Projection management

The various gRPC client APIs include dedicated clients that allow you to manage projections.

Warning

Currently not all clients fully expose all operations.

For a detailed explanation of projections, see the server documentation.

You can find the full sample code from this documentation page in the respective clients repositories.

Required packages

Install the client SDK package to your project.

NodeJS

Add @eventstore/db-client to your project using Yarn, NPM, or pnpm.

TypeScript type declarations are included in the package.

npm

npm install --save @eventstore/db-client

yarn

yarn add @eventstore/db-client

pnpm

pnpm add @eventstore/db-client

Java

Add the db-client-java dependency to your project using Maven or Gradle.

Maven

<dependency>
  <groupId>com.eventstore</groupId>
  <artifactId>db-client-java</artifactId>
  <version>5.2.0</version>
</dependency>

Gradle

implementation 'com.eventstore:db-client-java:5.2.0'

.NET

For .NET projects, install the EventStore.Client.Grpc.ProjectionManagement package from NuGet. It is not the same package as the regular client package.

dotnet add package EventStore.Client.Grpc.ProjectionManagement --version 23.*

Other languages

Go, Rust and Python clients does not have a dedicated projection management client yet.

Creating a client

Projection management operations are exposed through a dedicated client.

JavaScript

const client = EventStoreDBClient.connectionString`
  esdb+discover://${ADMIN}:${PASSWORD}@${ENDPOINT}?nodePreference=leader

TypeScript

const client = EventStoreDBClient.connectionString`
  esdb+discover://${ADMIN}:${PASSWORD}@${ENDPOINT}?nodePreference=leader

Python

client = EventStoreDBClient(
    uri="{connectionString}"
)

Java

EventStoreDBClientSettings settings = EventStoreDBConnectionString.parseOrThrow(connection);
EventStoreDBProjectionManagementClient client = EventStoreDBProjectionManagementClient.create(settings);

C#

var settings = EventStoreClientSettings.Create(connection);
settings.ConnectionName     = "Projection management client";
settings.DefaultCredentials = new UserCredentials("admin", "changeit");
var managementClient = new EventStoreProjectionManagementClient(settings);

Create a projection

Creates a projection that runs until the last event in the store, and then continues processing new events as they are appended to the store. The query parameter contains the JavaScript you want created as a projection. Projections have explicit names, and you can enable or disable them via this name.

JavaScript

const name = `countEvents_Create_${uuid()}`;
const projection = `
  fromAll()
      .when({
          $init() {
              return {
                  count: 0,
              };
          },
          $any(s, e) {
              s.count += 1;
          }
      })
      .outputState();

await client.createProjection(name, projection);

TypeScript

const name = `countEvents_Create_${uuid()}`;
const projection = `
  fromAll()
      .when({
          $init() {
              return {
                  count: 0,
              };
          },
          $any(s, e) {
              s.count += 1;
          }
      })
      .outputState();

await client.createProjection(name, projection);

Java

String js =
        "fromAll()" +
        ".when({" +
        "    $init: function() {" +
        "        return {" +
        "            count: 0" +
        "        };" +
        "    }," +
        "    $any: function(s, e) {" +
        "        s.count += 1;" +
        "    }" +
        "})" +
        ".outputState();";

String name = "countEvents_Create_" + java.util.UUID.randomUUID();

client.create(name, js).get();

C#

const string js = @"fromAll()
                            .when({
                                $init: function() {
                                    return {
                                        count: 0
                                    };
                                },
                                $any: function(s, e) {
                                    s.count += 1;
                                }
                            })
                            .outputState();";

var name = $"countEvents_Create_{Guid.NewGuid()}";
await managementClient.CreateContinuousAsync(name, js);

Trying to create projections with the same name will result in an error:

JavaScript

try {
  await client.createProjection(name, projection);
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("Conflict"))
    throw err;
  console.log(`${name} already exists`);
}

TypeScript

try {
  await client.createProjection(name, projection);
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("Conflict"))
    throw err;
  console.log(`${name} already exists`);
}

Java

try {
    client.create(name, js).get();
} catch (ExecutionException ex) {
    if (ex.getMessage().contains("Conflict")) {
        System.out.println(name + " already exists");
    }
}

C#

await managementClient.CreateContinuousAsync(name, js);
try {
    await managementClient.CreateContinuousAsync(name, js);
}
catch (RpcException e) when (e.StatusCode is StatusCode.AlreadyExists) {
    Console.WriteLine(e.Message);
}
catch (RpcException e) when (e.Message.Contains("Conflict")) { // will be removed in a future release
    var format = $"{name} already exists";
    Console.WriteLine(format);
}

Restart the subsystem

It is possible to restart the entire projection subsystem using the projections management client API. The user must be in the $ops or $admin group to perform this operation.

JavaScript

await client.restartSubsystem();

TypeScript

await client.restartSubsystem();

Java

client.restartSubsystem().get();

C#

await managementClient.RestartSubsystemAsync();

Enable a projection

Enables an existing projection by name. Once enabled, the projection will start to process events even after restarting the server or the projection subsystem. You must have access to a projection to enable it, see the ACL documentation.

JavaScript

await client.enableProjection("$by_category");

TypeScript

await client.enableProjection("$by_category");

Java

client.enable("$by_category").get();

C#

await managementClient.EnableAsync("$by_category");

You can only enable an existing projection. When you try to enable a non-existing projection, you'll get an error:

JavaScript

const projectionName = "projection that does not exist";

try {
  await client.enableProjection(projectionName);
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("NotFound"))
    throw err;
  console.log(`${projectionName} does not exist`);
}

TypeScript

const projectionName = "projection that does not exist";

try {
  await client.enableProjection(projectionName);
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("NotFound"))
    throw err;
  console.log(`${projectionName} does not exist`);
}

Java

try {
    client.disable("projection that does not exists").get();
} catch (ExecutionException ex) {
    if (ex.getMessage().contains("NotFound")) {
        System.out.println(ex.getMessage());
    }
}

C#

try {
    await managementClient.EnableAsync("projection that does not exists");
}
catch (RpcException e) when (e.StatusCode is StatusCode.NotFound) {
    Console.WriteLine(e.Message);
}
catch (RpcException e) when (e.Message.Contains("NotFound")) { // will be removed in a future release
    Console.WriteLine(e.Message);
}

Disable a projection

Disables a projection, this will save the projection checkpoint. Once disabled, the projection will not process events even after restarting the server or the projection subsystem. You must have access to a projection to disable it, see the ACL documentation.

JavaScript

await client.disableProjection("$by_category");

TypeScript

await client.disableProjection("$by_category");

Java

client.disable("$by_category").get();

C#

await managementClient.DisableAsync("$by_category");

You can only disable an existing projection. When you try to disable a non-existing projection, you'll get an error:

JavaScript

const projectionName = "projection that does not exist";

try {
  await client.disableProjection(projectionName);
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("NotFound"))
    throw err;
  console.log(`${projectionName} does not exist`);
}

TypeScript

const projectionName = "projection that does not exist";

try {
  await client.disableProjection(projectionName);
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("NotFound"))
    throw err;
  console.log(`${projectionName} does not exist`);
}

Java

try {
    client.disable("projection that does not exists").get();
} catch (ExecutionException ex) {
    if (ex.getMessage().contains("NotFound")) {
        System.out.println(ex.getMessage());
    }
}

C#

try {
    await managementClient.DisableAsync("projection that does not exists");
}
catch (RpcException e) when (e.StatusCode is StatusCode.NotFound) {
    Console.WriteLine(e.Message);
}
catch (RpcException e) when (e.Message.Contains("NotFound")) { // will be removed in a future release
    Console.WriteLine(e.Message);
}

Delete a projection

Deletes an existing projection. You must disable the projection before deleting it, running projections cannot be deleted. Deleting a projection includes deleting the checkpoint and the emitted streams.

JavaScript

// A projection must be disabled to allow it to be deleted.
await client.disableProjection(name);

// The projection can now be deleted
await client.deleteProjection(name);

TypeScript

// A projection must be disabled to allow it to be deleted.
await client.disableProjection(name);

// The projection can now be deleted
await client.deleteProjection(name);

Java

// A projection must be disabled to allow it to be deleted.
client.disable(name).get();
// The projection can now be deleted
client.delete(name).get();

C#

// this is not yet available in the .net grpc client

You can only delete an existing projection. When you try to delete a non-existing projection, you'll get an error:

JavaScript

const projectionName = "projection that does not exist";

try {
  await client.deleteProjection(projectionName);
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("NotFound"))
    throw err;
  console.log(`${projectionName} does not exist`);
}

TypeScript

const projectionName = "projection that does not exist";

try {
  await client.deleteProjection(projectionName);
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("NotFound"))
    throw err;
  console.log(`${projectionName} does not exist`);
}

Java

try {
    client.delete("projection that does not exists").get();
} catch (ExecutionException ex) {
    if (ex.getMessage().contains("NotFound")) {
        System.out.println(ex.getMessage());
    }
}

Abort a projection

Aborts a projection, this will not save the projection's checkpoint.

JavaScript

await client.abortProjection(name);

TypeScript

await client.abortProjection(name);

Java

client.abort("$by_category").get();

C#

// The .net clients prior to version 21.6 had an incorrect behavior: they will save the checkpoint.
await managementClient.AbortAsync("countEvents_Abort");

You can only abort an existing projection. When you try to abort a non-existing projection, you'll get an error:

JavaScript

const projectionName = "projection that does not exist";

try {
  await client.abortProjection(projectionName);
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("NotFound"))
    throw err;
  console.log(`${projectionName} does not exist`);
}

TypeScript

const projectionName = "projection that does not exist";

try {
  await client.abortProjection(projectionName);
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("NotFound"))
    throw err;
  console.log(`${projectionName} does not exist`);
}

Java

try {
    client.abort("projection that does not exists").get();
} catch (ExecutionException ex) {
    if (ex.getMessage().contains("NotFound")) {
        System.out.println(ex.getMessage());
    }
}

C#

try {
    await managementClient.AbortAsync("projection that does not exists");
}
catch (RpcException e) when (e.StatusCode is StatusCode.NotFound) {
    Console.WriteLine(e.Message);
}
catch (RpcException e) when (e.Message.Contains("NotFound")) { // will be removed in a future release
    Console.WriteLine(e.Message);
}

Reset a projection

Resets a projection, which causes deleting the projection checkpoint. This will force the projection to start afresh and re-emit events. Streams that are written to from the projection will also be soft-deleted.

JavaScript

await client.resetProjection(name);

TypeScript

await client.resetProjection(name);

Java

client.reset("$by_category").get();

C#

// Checkpoint will be written prior to resetting the projection
await managementClient.ResetAsync("countEvents_Reset");

Resetting a projection that does not exist will result in an error.

JavaScript

const projectionName = "projection that does not exist";

try {
  await client.resetProjection(projectionName);
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("NotFound"))
    throw err;
  console.log(`${projectionName} does not exist`);
}

TypeScript

const projectionName = "projection that does not exist";

try {
  await client.resetProjection(projectionName);
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("NotFound"))
    throw err;
  console.log(`${projectionName} does not exist`);
}

Java

try {
    client.reset("projection that does not exists").get();
} catch (ExecutionException ex) {
    if (ex.getMessage().contains("NotFound")) {
        System.out.println(ex.getMessage());
    }
}

C#

try {
    await managementClient.ResetAsync("projection that does not exists");
}
catch (RpcException e) when (e.StatusCode is StatusCode.NotFound) {
    Console.WriteLine(e.Message);
}
catch (RpcException e) when (e.Message.Contains("NotFound")) { // will be removed in a future release
    Console.WriteLine(e.Message);
}

Update a projection

Updates a projection with a given name. The query parameter contains the new JavaScript. Updating system projections using this operation is not supported at the moment.

JavaScript

const name = `countEvents_Update_${uuid()}`;
const projection = `
  fromAll()
      .when({
          $init() {
              return {
                  count: 0,
              };
          },
          $any(s, e) {
              s.count += 1;
          }
      })
      .outputState();

await client.createProjection(name, "fromAll().when()");
await client.updateProjection(name, projection);

TypeScript

const name = `countEvents_Update_${uuid()}`;
const projection = `
  fromAll()
      .when({
          $init() {
              return {
                  count: 0,
              };
          },
          $any(s, e) {
              s.count += 1;
          }
      })
      .outputState();

await client.createProjection(name, "fromAll().when()");
await client.updateProjection(name, projection);

Java

String name = "countEvents_Update_" + java.util.UUID.randomUUID();
String js =
        "fromAll()" +
                ".when({" +
                "    $init: function() {" +
                "        return {" +
                "            count: 0" +
                "        };" +
                "    }," +
                "    $any: function(s, e) {" +
                "        s.count += 1;" +
                "    }" +
                "})" +
                ".outputState();";

client.create(name, "fromAll().when()").get();
client.update(name, js).get();

C#

const string js = @"fromAll()
                            .when({
                                $init: function() {
                                    return {
                                        count: 0
                                    };
                                },
                                $any: function(s, e) {
                                    s.count += 1;
                                }
                            })
                            .outputState();";

var name = $"countEvents_Update_{Guid.NewGuid()}";

await managementClient.CreateContinuousAsync(name, "fromAll().when()");
await managementClient.UpdateAsync(name, js);

You can only update an existing projection. When you try to update a non-existing projection, you'll get an error:

JavaScript

const projectionName = "projection that does not exist";

try {
  await client.updateProjection(projectionName, "fromAll().when()");
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("NotFound"))
    throw err;
  console.log(`${projectionName} does not exist`);
}

TypeScript

const projectionName = "projection that does not exist";

try {
  await client.updateProjection(projectionName, "fromAll().when()");
} catch (err) {
  if (!isCommandError(err) || !err.message.includes("NotFound"))
    throw err;
  console.log(`${projectionName} does not exist`);
}

Java

try {
    client.update("Update Not existing projection", "fromAll().when()").get();
} catch (ExecutionException ex) {
    if (ex.getMessage().contains("NotFound")) {
        System.out.println("'Update Not existing projection' does not exists and can not be updated");
    }
}

C#

try {
    await managementClient.UpdateAsync("Update Not existing projection", "fromAll().when()");
}
catch (RpcException e) when (e.StatusCode is StatusCode.NotFound) {
    Console.WriteLine(e.Message);
}
catch (RpcException e) when (e.Message.Contains("NotFound")) { // will be removed in a future release
    Console.WriteLine("'Update Not existing projection' does not exists and can not be updated");
}

List all projections

Returns a list of all projections, user defined & system projections. See the projection details section for an explanation of the returned values.

JavaScript

// This is currently not available in the nodejs client

TypeScript

// This is currently not available in the nodejs client

Java

List<ProjectionDetails> details = client.list().get();

for (ProjectionDetails detail: details) {
    System.out.println(
        detail.getName() + ", " +
        detail.getStatus() + ", " +
        detail.getCheckpointStatus() + ", " +
        detail.getMode() + ", " +
        detail.getProgress());
}

C#

var details = managementClient.ListAllAsync();
await foreach (var detail in details)
    Console.WriteLine(
        $@"{detail.Name}, {detail.Status}, {detail.CheckpointStatus}, {detail.Mode}, {detail.Progress}"
    );

List continuous projections

Returns a list of all continuous projections. See the projection details section for an explanation of the returned values.

JavaScript

const projections = await client.listProjections();

for (const { name, status, checkpointStatus, progress } of projections) {
  console.log(name, status, checkpointStatus, progress);
}

TypeScript

const projections = await client.listProjections();

for (const { name, status, checkpointStatus, progress } of projections) {
  console.log(name, status, checkpointStatus, progress);
}

Java

List<ProjectionDetails> details = client.list().get();

for (ProjectionDetails detail: details) {
    System.out.println(
        detail.getName() + ", " +
        detail.getStatus() + ", " +
        detail.getCheckpointStatus() + ", " +
        detail.getMode() + ", " +
        detail.getProgress());
}

C#

var details = managementClient.ListContinuousAsync();
await foreach (var detail in details)
    Console.WriteLine(
        $@"{detail.Name}, {detail.Status}, {detail.CheckpointStatus}, {detail.Mode}, {detail.Progress}"
    );

Get status

Gets the status of a named projection. See the projection details section for an explanation of the returned values.

JavaScript

const projection = await client.getProjectionStatus(name);

console.log(
  projection.name,
  projection.status,
  projection.checkpointStatus,
  projection.progress
);

TypeScript

const projection = await client.getProjectionStatus(name);

console.log(
  projection.name,
  projection.status,
  projection.checkpointStatus,
  projection.progress
);

Java

ProjectionDetails status = client.getStatus("$by_category").get();
System.out.println(
    status.getName() + ", " +
    status.getStatus() + ", " +
    status.getCheckpointStatus() + ", " +
    status.getMode() + ", " +
    status.getProgress());

C#

await managementClient.CreateContinuousAsync(name, js);
var status = await managementClient.GetStatusAsync(name);
Console.WriteLine(
    $@"{status?.Name}, {status?.Status}, {status?.CheckpointStatus}, {status?.Mode}, {status?.Progress}"
);

Get state

Retrieves the state of a projection.

JavaScript

  const name = `get_state_example`;
  const projection = `
    fromAll()
        .when({
            $init() {
                return {
                    count: 0,
                };
            },
            $any(s, e) {
                s.count += 1;
            }
        })
        .transformBy((state) => state.count)
        .outputState();
`;

  await client.createProjection(name, projection);

  // Give it some time to count event
  await delay(500);

  const state = await client.getProjectionState(name);

  console.log(`Counted ${state.count} events.`);

TypeScript

interface CountProjectionState {
  count: number;
}

const name = `get_state_example`;
const projection = `
  fromAll()
      .when({
          $init() {
              return {
                  count: 0,
              };
          },
          $any(s, e) {
              s.count += 1;
          }
      })
      .transformBy((state) => state.count)
      .outputState();


await client.createProjection(name, projection);

// Give it some time to count event
await delay(500);

const state = await client.getProjectionState<CountProjectionState>(name);

console.log(`Counted ${state.count} events.`);

Java

// This example requires the following class to be defined:
//
//    public static class CountResult {
//        private int count;
//        public int getCount() {
//            return count;
//        }
//        public void setCount(final int count){
//            this.count = count;
//        }
//    }
String name = "get_state_example";
String js =
    "fromAll()" +
    ".when({" +
    "    $init() {" +
    "        return {" +
    "            count: 0," +
    "        };" +
    "    }," +
    "    $any(s, e) {" +
    "        s.count += 1;" +
    "    }" +
    "})" +
    ".outputState();";

client.create(name, js).get();

Thread.sleep(500); //give it some time to process and have a state.

CountResult result = client
    .getState(name, CountResult.class)
    .get();

System.out.println(result);

C#

const string js =
    "fromAll().when({$init:function(){return {count:0};},$any:function(s, e){s.count += 1;}}).outputState();";

var name = $"countEvents_State_{Guid.NewGuid()}";

await managementClient.CreateContinuousAsync(name, js);
//give it some time to process and have a state.
await Task.Delay(500);

var stateDocument = await managementClient.GetStateAsync(name);
var result        = await managementClient.GetStateAsync<Result>(name);

Console.WriteLine(DocToString(stateDocument));
Console.WriteLine(result);

static async Task<string> DocToString(JsonDocument d) {
    await using var stream = new MemoryStream();
    var             writer = new Utf8JsonWriter(stream, new JsonWriterOptions { Indented = false });
    d.WriteTo(writer);
    await writer.FlushAsync();
    return Encoding.UTF8.GetString(stream.ToArray());
}

Get result

Retrieves the result of the named projection and partition.

JavaScript

const name = `get_result_example`;
const projection = `
  fromAll()
      .when({
          $init() {
              return {
                  count: 0,
              };
          },
          $any(s, e) {
              s.count += 1;
          }
      })
      .transformBy((state) => state.count)
      .outputState();


await client.createProjection(name, projection);

// Give it some time to have a result.
await delay(500);

const result = await client.getProjectionResult(name);

console.log(`Counted ${result} events.`);

TypeScript

const name = `get_result_example`;
const projection = `
  fromAll()
      .when({
          $init() {
              return {
                  count: 0,
              };
          },
          $any(s, e) {
              s.count += 1;
          }
      })
      .transformBy((state) => state.count)
      .outputState();


await client.createProjection(name, projection);

// Give it some time to have a result.
await delay(500);

const result = await client.getProjectionResult<number>(name);

console.log(`Counted ${result} events.`);

Java

String name = "get_result_example";
String js =
    "fromAll()" +
    ".when({" +
    "    $init() {" +
    "        return {" +
    "            count: 0," +
    "        };" +
    "    }," +
    "    $any(s, e) {" +
    "        s.count += 1;" +
    "    }" +
    "})" +
    ".transformBy((state) => state.count)" +
    ".outputState();";

client.create(name, js).get();

Thread.sleep(500); //give it some time to process and have a state.

int result = client
        .getResult(name, int.class)
        .get();

System.out.println(result);

C#

const string js = @"fromAll()
                            .when({
                                $init: function() {
                                    return {
                                        count: 0
                                    };
                                },
                                $any: function(s, e) {
                                    s.count += 1;
                                }
                            })
                            .outputState();";

var name = $"countEvents_Result_{Guid.NewGuid()}";

await managementClient.CreateContinuousAsync(name, js);
await Task.Delay(500); //give it some time to have a result.

// Results are retrieved either as  JsonDocument or a typed result 
var document = await managementClient.GetResultAsync(name);
var result   = await managementClient.GetResultAsync<Result>(name);

Console.WriteLine(DocToString(document));
Console.WriteLine(result);

static string DocToString(JsonDocument d) {
    using var stream = new MemoryStream();
    using var writer = new Utf8JsonWriter(stream, new JsonWriterOptions { Indented = false });
    d.WriteTo(writer);
    writer.Flush();
    return Encoding.UTF8.GetString(stream.ToArray());
}

Projection Details

List all, list continuous and get status all return the details and statistics of projections

Field	Description
Name, EffectiveName	The name of the projection
Status	A human readable string of the current statuses of the projection (see below)
StateReason	A human readable string explaining the reason of the current projection state
CheckpointStatus	A human readable string explaining the current operation performed on the checkpoint : requested, writing
Mode	Continuous, OneTime , Transient
CoreProcessingTime	The total time, in ms, the projection took to handle events since the last restart
Progress	The progress, in %, indicates how far this projection has processed event, in case of a restart this could be -1% or some number. It will be updated as soon as a new event is appended and processed
WritesInProgress	The number of write requests to emitted streams currently in progress, these writes can be batches of events
ReadsInProgress	The number of read requests currently in progress
PartitionsCached	The number of cached projection partitions
Position	The Position of the last processed event
LastCheckpoint	The Position of the last checkpoint of this projection
EventsProcessedAfterRestart	The number of events processed since the last restart of this projection
BufferedEvents	The number of events in the projection read buffer
WritePendingEventsBeforeCheckpoint	The number of events waiting to be appended to emitted streams before the pending checkpoint can be written
WritePendingEventsAfterCheckpoint	The number of events to be appended to emitted streams since the last checkpoint
Version	This is used internally, the version is increased when the projection is edited or reset
Epoch	This is used internally, the epoch is increased when the projection is reset

The Status string is a combination of the following values. The first 3 are the most common one, as the other one are transient values while the projection is initialised or stopped

Value	Description
Running	The projection is running and processing events
Stopped	The projection is stopped and is no longer processing new events
Faulted	An error occurred in the projection, StateReason will give the fault details, the projection is not processing events
Initial	This is the initial state, before the projection is fully initialised
Suspended	The projection is suspended and will not process events, this happens while stopping the projection
LoadStateRequested	The state of the projection is being retrieved, this happens while the projection is starting
StateLoaded	The state of the projection is loaded, this happens while the projection is starting
Subscribed	The projection has successfully subscribed to its readers, this happens while the projection is starting
FaultedStopping	This happens before the projection is stopped due to an error in the projection
Stopping	The projection is being stopped
CompletingPhase	This happens while the projection is stopping
PhaseCompleted	This happens while the projection is stopping