Reactive DataSource for Angular

This article presents you @matheo/datasource which aims to facilitate the listing of any kind of data inside a Material Table or any Angular Component. It’s a battle-tested library but improvements and suggestions are warmly welcomed. This article is the initial documentation as I need some help and time to build a decent one :D

I’ve mounted a demo app with a (hopefully) self-explanatory list of commits, building the list step by step, consuming a simple Firestore collection:

Screenshot of the demo list that will showcase the features

The Concepts

Some years ago I had the challenge to build different lists with mat-table, so I took some code snippets from the official documentation (thanks for the awesome docs material team!) and I saw their approach having a Database service providing the data, to a DataSource that was consumed by the table component on initialization, via aconnect() method.

The connect() returns a stream of data to be listed. Nowadays, the mat-table.dataSource input can be an array of data, a direct stream or a DataSource instance implementing the connect/disconnect interface.

I loved the DataSource approach and I started to dream with a flexible class to build data lists with ease, reusing it in the different scenarios that I had to implement. In those days I had no experience with rxjs and I had to feed this DataSource with different observables from the UI events like Sorting, Pagination, Filtering, and sometimes System events.

That stream of events flows within this structure:

• a UI triggering different events for the List,
• a DataSource collects them to build a REQuest object,
• a Database processing the request into a RAW data result,
• a mat-table consuming a post-processed RESult fromconnect.

the upper-case notation corresponds to the MatDataSource definition:

abstract class MatDataSource<REQ, RAW, RES> extends DataSource<RES>

The Code

Start installing the library as usual:

yarn add @matheo/datasource or npm install @matheo/datasource

and adding MatDataSourceModule to your modules aimed to list data:

import { MatDataSourceModule } from '@matheo/datasource';

Database

The database service will translate a REQuest into a RAW list of data to be processed by the DataSource and obtain a RESult, so, the database service methods are queries to the backend or any source of data.

Simplest Firebase Database

It can be as simple as in the image, or with some complexity, it will query the backend considering each scenario and will look like this.
This simple database is not processing a REQuest but listing some items, and with this initial database, you will be able to see your table working and start to extend it depending on the required functionality as you can see in the commit list.

DataSource

The datasource service will collect (register) the change-event input streams via the addStream method; each stream emits a full or partial REQuest object, some of them might need to startWith a default value. For instance, MatPaginator.page observable streams a PageEvent that can be formatted to match your pagination parameters, and by default it uses the configuredpageIndex and pageSize.

Once we use the connect method, the datasource will combine the registered streams and merges their output values to produce the REQuest object.

We can configure it to know when to autoStart or not, so it will block the stream if necessary; also, it will skip the repeated REQuests, and if you want to force it you can use the reload method. After that, it will execute the query, process the total count, post-process the RAW data, and returns an array of RESulting items.

There are some additional processes under the hood, like timeout messages if there’s no response after some seconds, updating the isLoading, isLoaded, isEmpty state flags and triggering the change$ observable every time the DataSource changes its state.

To implement your DataSources you only have to customize the core processing of your data, from building up the REQuest object, fetching the RAW data, having a default RAW response in case of error, fetching the total count for pagination purposes, and post-processing the RESult to get your desired data on the table!

The DataSource for Firestore required some special considerations to handle the sorting and the total, and the queries of course. The complete code can be seen here, and any feedback is welcome :)

List Component

Personally, I do provide the Database service globally (providedIn: 'root') but the DataSource locally from the container component, so I don’t have conflicts with another instance configured in a different way. I include it in the providers of the container component and pass it to child components.

Also, the library provides an overlay component, to render the table and other elements and show the loading spinner, the configured timeout messages, and any error or empty message too. You just need to wrap the mat-table inside it:

<mat-datasource [dataSource]="source">
<mat-table [dataSource]="source" ...>...</mat-table>
</mat-datasource>

You can customize the look and feel vía the .mat-datasource-overlay CSS class which defaults to a flexbox with centered content, and use .mat-datasource-empty , .mat-datasource-error and .mat-datasource-loading for the configured messages.

You can configure the DataSource in the way you want vía the config API. I usually put the global config in the constructor, and some custom stuff inside each container component. Note that you can enable the debug to get verbose information of each step in the data processing.

Input Streams / Data Mutators

As mentioned, you add the streams to listen vía addStream. For the MatSort and MatPaginator streams, the library offers built-in methods to extract the relevant arguments, like the orderBy, orderDir, pageIndex and pageSize respectively, so you can simply use:

this.dataSource.setPaginator(this.paginator)

where paginator is the MatPaginator ViewChild of the component.

Additional streams can be registered like this, listening form changes, and setting up the form.valueChanges stream to start with the form.value as the initial part of the REQuest. The DataSource merges all the outputs of the streams to build the complete REQuest object.

Furthermore

This DataSource can be used in Autocompleters and Search components extending the ReactiveDatasource abstract class, and implementing two additional methods:

• filter to search a certain query and get a limit set of items:
  filter(query: string, limit: number): void
• resFilter to post-process the matching results and format them in a friendly standard format for item lists or auto-completers:
  resFilter(result: RES[]): DataSourceItem[]

Also, there’s a dataSource pipe to work like the async pipe but taking a ReactiveDataSource as input: *ngIf="source | dataSource as items".

Demo Running

Running example with Sort, Pagination, and Filtering.

Enjoy!

That’s it! basically. I need to get some time to build the documentation (help is welcome!) and maybe share some Databases for different needs, like I did this time with the Firestore specific requirements.

Please leave a comment with your impression,
even if it’s to say that I over-extended the size of this article,
or improvements in the writing (apologies in advance)
Have fun! :D