# Server-side filtering

EventStoreDB allows you to filter the events whilst you subscribe to the $all stream so that you only receive the events that you care about.

You can filter by event type or stream name using either a regular expression or a prefix. Server-side filtering is currently only available on the $all stream.

TIP

Server-side filtering introduced as a simpler alternative to projections. Before creating a projection to get the events you care about you should first consider filtering.

# Filtering out system events

There are a number of events in EventStoreDB called system events. These are prefixed with a $ and under most circumstances you won't care about these. They can be filtered out by passing in a SubscriptionFilterOptions when subscribing to the $all stream.

    await client.SubscribeToAllAsync(Position.Start,
    	(s, e, c) => {
    		Console.WriteLine(
    			$"{e.Event.EventType} @ {e.Event.Position.PreparePosition}");
    		return Task.CompletedTask;
    	},
    	filterOptions: new SubscriptionFilterOptions(
    		EventTypeFilter.ExcludeSystemEvents())
    );
    SubscriptionListener listener = new SubscriptionListener() {
        @Override
        public void onEvent(Subscription subscription, ResolvedEvent event) {
            System.out.println("Received event"
                    + event.getOriginalEvent().getStreamRevision().getValueUnsigned()
                    + "@" + event.getOriginalEvent().getStreamId());
        }
    };
    String excludeSystemEventsRegex = "/^[^\\$].*/";
    
    SubscriptionFilter filter = SubscriptionFilter.newBuilder()
            .withEventTypeRegularExpression(excludeSystemEventsRegex)
            .build();
    
    SubscribeToAllOptions options = SubscribeToAllOptions.get()
            .filter(filter);
    
    client.subscribeToAll(
            listener,
            options
    );
    const subscription = client
      .subscribeToAll({
        fromRevision: START,
        filter: excludeSystemEvents(),
      })
      .on("data", function (resolvedEvent) {
        console.log(
          `Received event ${resolvedEvent.event.revision}@${resolvedEvent.event.streamId}`
        );
      });
    let filter = SubscriptionFilter::on_event_type().regex("/^[^\\$].*/");
    let options = SubscribeToAllOptions::default().filter(filter);
    
    let mut sub = client.subscribe_to_all(&options).await?;
    
    while let Some(event) = sub.try_next().await? {
        if let SubEvent::EventAppeared(event) = event {
            let stream_id = event.get_original_stream_id();
            let revision = event.get_original_event().revision;
    
            println!("Received event {}@{}", revision, stream_id);
        }
    }
    // Make sure to add code blocks to your code group

    TIP

    $stats events are no longer stored in EventStoreDB by default so there won't be as many $ events as before.

    # Filtering by event type

    If you only want to subscribe to events of a given type there are two options. You can either use a regular expression or a prefix.

    # Filtering by prefix

    If you want to filter by prefix pass in a SubscriptionFilterOptions to the subscription with an EventTypeFilter.Prefix.

      var filter = new SubscriptionFilterOptions(
      	EventTypeFilter.Prefix("customer-"));
      SubscriptionFilter filter = SubscriptionFilter.newBuilder()
              .withEventTypePrefix("customer-")
              .build();
      const prefixes = ["customer-"];
      const filter = eventTypeFilter({ prefixes });
      let filter = SubscriptionFilter::on_event_type().add_prefix("customer-");
      let options = SubscribeToAllOptions::default().filter(filter);
      
      let mut sub = client.subscribe_to_all(&options).await?;
      // Make sure to add code blocks to your code group

      This will only subscribe to events with a type that begin with customer-.

      # Filtering by regular expression

      If you want to subscribe to multiple event types then it might be better to provide a regular expression.

        var filter = new SubscriptionFilterOptions(
        	EventTypeFilter.RegularExpression("^user|^company"));
        SubscriptionFilter filter = SubscriptionFilter.newBuilder()
                .withEventTypeRegularExpression("^user|^company")
                .build();
        const regex = /^user|^company/;
        const filter = eventTypeFilter({ regex });
        let filter = SubscriptionFilter::on_event_type().regex("^user|^company");
        let options = SubscribeToAllOptions::default().filter(filter);
        
        let mut sub = client.subscribe_to_all(&options).await?;
        // Make sure to add code blocks to your code group

        This will subscribe to any event that begins with user or company.

        # Filtering by stream name

        If you only want to subscribe to streams with a given name there are two options. You can either use a regular expression or a prefix.

        # Filtering by prefix

        If you want to filter by prefix pass in a SubscriptionFilterOptions to the subscription with an StreamFilter.Prefix.

          var filter = new SubscriptionFilterOptions(
          	StreamFilter.Prefix("user-"));
          SubscriptionFilter filter = SubscriptionFilter.newBuilder()
                  .withStreamNamePrefix("user-")
                  .build();
          const prefixes = ["user-"];
          const filter = streamNameFilter({ prefixes });
          let filter = SubscriptionFilter::on_stream_name().add_prefix("user-");
          let options = SubscribeToAllOptions::default().filter(filter);
          
          let mut sub = client.subscribe_to_all(&options).await?;
          // Make sure to add code blocks to your code group

          This will only subscribe to streams with a name that begin with user-.

          # Filtering by regular expression

          If you want to subscribe to multiple streams then it might be better to provide a regular expression.

            var filter = new SubscriptionFilterOptions(
            	StreamFilter.RegularExpression("^account|^savings"));
            SubscriptionFilter filter = SubscriptionFilter.newBuilder()
                    .withStreamNameRegularExpression("^account|^savings")
                    .build();
            const regex = /^account|^savings/;
            const filter = streamNameFilter({ regex });
            let filter = SubscriptionFilter::on_event_type().regex("/^[^\\$].*/");
            let options = SubscribeToAllOptions::default().filter(filter);
            
            let mut sub = client.subscribe_to_all(&options).await?;
            // Make sure to add code blocks to your code group

            This will subscribe to any stream with a name that begins with account or savings.

            # Checkpointing

            There is one thing to consider with server-side filtering, and that is when events that match your filter are few and far between. In this scenario, you might find yourself in the situation where EventStoreDB has searched through 1 million events, and the last thing you want to happen is for the server to get to event 900k and then have your client crash. It won't have been able to take a checkpoint and upon a restart, you'd have to go back to the beginning and start again.

            In this case you can make use of an additional delegate that will be triggered every n number of events (32 by default).

            To make use of it set up checkpointReached on the SubscriptionFilterOptions class.

              var filter = new SubscriptionFilterOptions(
              	EventTypeFilter.ExcludeSystemEvents(),
              	checkpointReached: (s, p, c) =>
              	{
              		Console.WriteLine($"checkpoint taken at {p.PreparePosition}");
              		return Task.CompletedTask;
              	});
              String excludeSystemEventsRegex = "/^[^\\$].*/";
              
              SubscriptionFilter filter = SubscriptionFilter.newBuilder()
                      .withEventTypeRegularExpression(excludeSystemEventsRegex)
                      .withCheckpointer(
                              new Checkpointer() {
                                  @Override
                                  public CompletableFuture<Void> onCheckpoint(Subscription subscription, Position position) {
                                      System.out.println("checkpoint taken at {p.PreparePosition}");
                                      return CompletableFuture.completedFuture(null);
                                  }
                              })
                      .build();
              // TODO
              
              while let Some(event) = sub.try_next().await? {
                  match event {
                      SubEvent::EventAppeared(event) => {
                          let stream_id = event.get_original_stream_id();
                          let revision = event.get_original_event().revision;
              
                          println!("Received event {}@{}", revision, stream_id);
                      }
              
                      SubEvent::Checkpoint(position) => {
                          println!("checkpoint taken at {}", position.prepare);
                      }
              
                      _ => {}
                  }
              }
              // Make sure to add code blocks to your code group

              This will be called every n number of events. If you want to be specific about the number of events threshold you can also pass that as a parameter.

                var filter = new SubscriptionFilterOptions(
                	EventTypeFilter.ExcludeSystemEvents(),
                	checkpointInterval: 1000,
                	checkpointReached: (s, p, c) =>
                	{
                		Console.WriteLine($"checkpoint taken at {p.PreparePosition}");
                		return Task.CompletedTask;
                	});
                String excludeSystemEventsRegex = "/^[^\\$].*/";
                
                SubscriptionFilter filter = SubscriptionFilter.newBuilder()
                        .withEventTypeRegularExpression(excludeSystemEventsRegex)
                        .withCheckpointer(
                                new Checkpointer() {
                                    @Override
                                    public CompletableFuture<Void> onCheckpoint(Subscription subscription, Position position) {
                                        System.out.println("checkpoint taken at {p.PreparePosition}");
                                        return CompletableFuture.completedFuture(null);
                                    }
                                },
                                1000)
                        .build();
                const excludeSystemEventsRegex = /^[^\$].*/;
                const filter = eventTypeFilter({
                  checkpointIntervalMul: 1000,
                  regex: excludeSystemEventsRegex,
                });
                let filter = SubscriptionFilter::on_event_type().regex("/^[^\\$].*/");
                let options = SubscribeToAllOptions::default().filter(filter);
                
                let mut sub = client.subscribe_to_all(&options).await?;
                // Make sure to add code blocks to your code group

                WARNING

                This number will be called every n * 32 events.

                Last Updated: 3/2/2021, 1:16:11 PM