Smallpearl - Paginated Responses in NgRx/Data

This is a quick post on how to support paginated GET responses in NgRx/Data. Like many other posts, done primarily with the objective of recording my research output while sharing it with others who might need it. (Quick google searches did not yield much on this front).

Background

I have a Django backend with a REST API that generates paginated responses. Essentially the responses look like this:

response = {
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
        {
            "id": 1,
            "firstName": "Peter",
            "lastName": "Parker"
        },
        {
            "id": 2,
            "firstName": "John",
            "lastName": "Smith"
        }
    ]
}

The key point here is the response.next field. It is set to the next page URL, if more data is available at the remote. Using this allows us to avoid have to keep track of a page variable (though page index can be derived from this string) and provide infinite scrolling experience in the front-end.

Out of the box NgRx/data does not support this kind of responses. However, it provides extensive customization facilities which can be used to to support responses such as above.

How to do it

The class within NgRx/data that issues the actual REST API requests and handles its response is Entity DataService, the default implemention of which is provided as DefaultDataService<T>. As will be obvious from its name, it's a generic class, a concrete version of which will be generated for each registered entity. These concrete classes are created by the factory class, DefaultDataServiceFactory.

So the solution is to first create a customized version of DefaultDataService<T>, and then provide a customized DefaultDataServiceFactory, which would create this custom DefaultDataService<T> instead of the default one. Then tell NgRx to use the custom class factory by registering it in the modules provide array.

Here's the full code:

/**
 * DataService class that handles Django REST Framework's paginated
 * responses.
 */
export class PaginatedDataService<T> extends DefaultDataService<T> {
  private nextPageUrl: string = '';
  private count = 0;

  constructor(
    entityName: string,
    http: HttpClient,
    httpUrlGenerator: HttpUrlGenerator,
    config?: DefaultDataServiceConfig
  ) {
    super(entityName, http, httpUrlGenerator, config);
    this.nextPageUrl = this.entitiesUrl;
    this.count = 0;
  }

  // Override to store nextUrl as well as map response.results.
  getAll(): Observable<T[]> {
    return this.execute('GET', this.entitiesUrl).pipe(
      tap(response => {
        this.nextPageUrl = response.next;
        this.count = response.count;
      }),
      map(response => response.results)
    );
  }

  // Override to store nextUrl as well as map response.results.
  getWithQuery(queryParams: QueryParams | string): Observable<T[]> {
    const qParams =
      typeof queryParams === 'string'
        ? { fromString: queryParams }
        : { fromObject: queryParams };
    const params = new HttpParams(qParams);
    return this.execute('GET', this.entitiesUrl, undefined, { params }).pipe(
      tap(response => {
        // response.next would include the queryparams.
        this.nextPageUrl = response.next;
        this.count = response.count;
      }),
      map(response => response.results)
    );
  }

  /**
   * Get next page of results. Or empty array if remote data is
   * exhausted.
   */
  getMore(): Observable<T[]> {
    if (!this.hasMore()) {
      // or throwError?
      return of([]);
    }

    return this.execute('GET', this.nextPageUrl).pipe(
      tap(response => {
        this.nextPageUrl = response.next;
        this.count = response.count;
      }),
      map(response => response.results)
    );
  }

  /**
   * Returns total number of objects
   */
  totalCount() {
    return this.count;
  }

  /**
   * Returns boolean indicating if there's more data at server.
   */
  hasMore(): boolean {
    return !!this.nextPageUrl;
  }
}

/**
 * Custom DataServiceFactory that creates PaginatedDataService<T>,
 * instead of the DefaultDataService<T>
 */
@Injectable()
export class CustomDataServiceFactory extends DefaultDataServiceFactory {
  constructor(
    http: HttpClient,
    httpUrlGenerator: HttpUrlGenerator,
    @Optional() config?: DefaultDataServiceConfig
  ) {
    super(http, httpUrlGenerator, config);
  }

  create<T>(entityName: string): EntityCollectionDataService<T> {
    return new PaginatedDataService<T>(
      entityName,
      this.http,
      this.httpUrlGenerator,
      this.config
    );
  }
}

@NgModule({
  imports: [
    StoreModule.forRoot({}, {}),
    EffectsModule.forRoot([]),
    EntityDataModule.forRoot(entityConfig),
    StoreDevtoolsModule.instrument({ maxAge: 25, logOnly: environment.production }),
  ],
  providers: [
    { provide: DefaultDataServiceFactory, useClass: CustomDataServiceFactory },
  ]
})
export class AppStoreModule {
  constructor() {}
}

Disclaimer

A word of caution. I only tested this briefly as I decided to not use NgRx/Data for my own needs at the moment. I had spun my own implementation of something similar earlier, which is simpler and importantly, quite straightforward to use without having to come to grips with all the NgRx terminology.

I spent couple of hours today morning looking to NgRx/Data to see if it can replace my own data service implementation. While it definitely can, for my immediate needs, it's a little too much learning, not to mention wading into uncharted waters, for me.

But this should provide adequate clues for someone who wants to seriously use NgRx/Data and is stuck at getting around the lack of pagination support in it.