Make A List View In Angular

Recently, I’m working on an solution to show vast amount of card in a list and scroll smoothly on this list. Further more, the list should show a timeline meter on the side to allow user navigate to any position quickly just drag or click the timeline. This feature pose a challenge to develop a new kind of UI component which able to show any amount of card in screen without impact the performance.

A tradition way to show large amount of data in a list is so called infinite list or lazy loading . A list loads data at 20-30 items per page until user reaches bottom of the list. then bumps the next page of data and appends to current list. This solution actually contributes to the initial loading performance but did not too much to the overall scrolling performance. Using this solution, user cannot quickly scroll to the last position of data. even worse, when you have loaded 2000-3000 items, your tab crashes.

Find Another Approach.

The reason why lazy loading or traditional infinite list doesn’t work is obvious. As long as you continue added items into your list, those DOM elements will exists until eats up all of your memory. So this solution is not fit for our requirements.

If you ever have some experience on Android or iOS development. You may find the solution which has developed for a long time. Yes, that’s ListView (or RecyclerView, GridView) in Android and UITableView in iOS. They both use a similar idea which keep a limited amount of item on the fly and trash the view which is out of current viewport. To reduce the performance expense on create new views, they also recycle the trashed views and reuse them on demand. So this idea is what I choose for solving this problem.

First, How does this solution work?

The core of this solution is only maintaining a limited number of views in the list and keep remove and added necessary views into list when user scrolling the list. To make this possible we need to know the height of each view before performing a layout to place those views. In Android, the framework using a measurement procedure, but at web, we don’t have that mechanism, the dimension of an DOM element is control by css along with the element parent. So measurement on DOM is hard to do, we don’t want to discuss this topic at this article. We use a predefined height for each row. By doing this, we simplify the calculation of layout.

Another requirement to make this solution possible is we need to know the total number of items of our list. This is usually not much hard to do. To make things simple, we also make a little change, we have all data loaded once, so we don’t need to write load on demand.

Once we prepared data and container finish its layout, we will get container height , number of rows, row height . these information is pretty enough for our list view to render a initial state.

Initial State

As you can see, our initial state only contains very limited number of elements. This is what we expect, another information we got from this picture is we have a scrollbar which indicates we have very long content to scroll. we use a straight forward method to calculate the total height of scroll content by simply multiply row height and item count. Here is the main DOM structure to make this possible.

1
2
3
4
5
6
7
<!-- infinite-list has a height of 100% viewport height this height will be used to determine how many views to render -->
<div class="infinite-list">
    <!-- infinite-list-holder has a height of all -->
    <div class="infinite-list-holder" style="height: 140000px;">
        <list-item-example></list-item-example>
    </div>
</div>

infinite-list is the container of visible views, it use a overflow-y: auto css to make the infinite-list-holder scrollable. When this structure is established, browser will render a scrollbar for infinite-list.

When user scroll down or up, the infinite-list-holder will scroll and together with its view content. meanwhile we listen to the scroll event to asynchronously calculate how far we have scrolled from initial position.

THE key part of the solution can keep a low memory usage is keep calculating every view position relative to viewport, add views which are about to visible in the viewport and remove views which is already out of viewport.

When content scrolling, remove and added views

This is really very intuitive solution. keep added and remove views as long as there are the right views in our viewport, user will feel like our list is filled with all of the data items. OK, this is simple. But how do we know which view is out of viewport, which view is about to add in viewport and how to place them in the correct position.

Do the layout

we already have some basic information about our container infinite-list, we know its width and height. we also get height of every row and total number of data. when we do layout at scrolling, we need one more information to know how much have we scrolled. This is not hard to obtain. In all browser, we can use element.scrollTop property to know the relative distance between top of the scrolling content and top of the wrapper content. (More detail, you can learn from MDN)

Calculate the index of top most view relative to our viewport, remove any views before that index.

1
let firstIndex = Math.floor(scrollTop / rowHeight);

Calculate the index of bottom most view relative to our viewport, remove any views after that index.

1
2
3
4
// we need this offset to complement the height of viewport. because a view may just cross the border of list-container
let firstViewOffset = scrollTop - firstIndex * rowHeight;
// containerHeight is equal to viewport height as we assumed
let lastIndex = Math.ceil((containerHeight + firstViewOffset) / rowHeight) + firstIndex;

Once we get the index range, we can remove view not inside this range and added views according this range. To added view at correct position, we need calculate its top y coordinate relative the list-holder. This calculation is more easy. We can calculate a view y position base on row height and data item index:

A view top y position relative to infinite-list-holder is rowHeight * index

The method we use to place our views on infinite-list-holder is using css3 translate3d and let each view’s position property be absolute. By using this method we get the hardware acceleration and not trigger any re-layout out side the infinite-list-holder. This is another important performance trick.

1
2
3
4
5
viewElement.style.transform = `translate3d(0, ${y}px, 0)`;
viewElement.style.webkitTransform = `translate3d(0, ${y}px, 0)`;
viewElement.style.width = `${containerWidth}px`;
viewElement.style.height = `${rowHeight}px`;
viewElement.style.position = 'absolute';

Once styles is applied. insert view into infinite-list-holder at proper position, browser will render the view for us.

This process will repeat whenever we need to do a layout operation. At the user aspect, the view will really scroll in the list.

Optimization.

So far so good, We have give a smooth scrolling experience to user using our real infinite-list .but currently we only create a very simple example which using a simple structure for item view. What if we have a very complex view to render. we need to rapidly remove and add DOM element to our list-holder, the view element may have many object to initialize. many listener to add. Creating new object especially DOM element object is expensive. Those views in our list are very the same structure, the only different is there content. so we decide to do some optimization to reuse the views.

To reuse view, we need modified our solution at two phase, remove and add phase. At remove phase, we don’t directly destroy that trashed view, we detach that view and move it into a recycle bin. we can create a class called RecycleBin to manage those recycled views. At add phase, we first try to retrieve view from RecycleBin, if we find the same view with the same index we expect, we can directly reattach it back to list-holder, because its content is the same. But if we don’t have that type of view, we can also pop a view from scrap views, and replace its binding data with correct data at certain index. If you use a MVVM framework, the framework will do the rest things. only when we don’t have scraped views from RecycleBin, we create a new view with data of certain index.

RecycleBin

Our modified version reduce the dom create and destroy, also reduce the new memory allocation and GC operation (though this is really depend on JS engine). A drawback of this modification is we increase a little more memory usage. But if you can write some simple strategy to control the total number of scrap views. No need to worry about run out of memory.

So far we have briefly explained how our solution work. But we haven’t explained how to implement in Angular.

The Angular Implementation

we only cover the Angular 4 and above, if you want to find an AngularJS (1.x) implementation. You can read the code from this repository to see how to implement this approach in Angular 1.5, although this is grid view which is a little complicated.

In Angular, things are not much different. We still use the theory we have explained before plus a little angular feature.

In Angular, there are component and directive ( include structural directive ) which can manipulate DOM as we need. Recall our feature requirement, Our list, in fact, has repeat structure which presents data in same form repeatedly. The idea flash in your mind first must be ngFor directive which is a structural directive can clone its content repeatedly. The difference between ngFor and our directive is ngFor render all DOM in the collection we don’t. So we use a similar directive called infiniteFor which has a very same usage.

1
2
3
<any *infiniteFor="let row of collection">
    {{row}}
</any>

By using these semantic we borrowed from ngFor, we can create a local variable binding to its template from the iterable collection In Angular we don’t have the concept scope but it is a very similar concept which using an object to store your local variable and bind to the template. We won’t use too much words to describe how ngFor implement. if you don’t understand and have interest in this, you can read official document: Structural Directive and ngFor API Guide

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
@Directive({
    selector: '[infiniteFor][infiniteForOf]'
})
export class InfiniteForOf<T> implements OnChanges, DoCheck {
    private _differ: IterableDiffer<T>;
    private _trackByFn: TrackByFunction<T>;

    // This is a major difference from ngFor directive, we actually need store the whole collection to a data structure (we use array)
    private _collection: any[];

    @Input() infiniteForOf: NgIterable<T>;

    @Input()
    set infiniteForTrackBy(fn: TrackByFunction<T>) {
        this._trackByFn = fn;
    }

    get infiniteForTrackBy(): TrackByFunction<T> {
        return this._trackByFn;
    }

    @Input()
    set infiniteForTemplate(value: TemplateRef<InfiniteRow>) {
        if (value) {
            this._template = value;
        }
    }

    constructor(private _infiniteList: InfiniteList,
                private _differs: IterableDiffers,
                private _template: TemplateRef<InfiniteRow>,
                private _viewContainerRef: ViewContainerRef) {
    }

    ngOnChanges(changes: SimpleChanges): void {
        if ('infiniteForOf' in changes) {
            // React on infiniteForOf only once all inputs have been initialized
            const value = changes['infiniteForOf'].currentValue;
            if (!this._differ && value) {
                try {
                    this._differ = this._differs.find(value).create(this._trackByFn);
                } catch (e) {
                    throw new Error(`Cannot find a differ supporting object '${value}' of type '${getTypeNameForDebugging(value)}'. NgFor only supports binding to Iterables such as Arrays.`);
                }
            }
        }
    }

    ngDoCheck(): void {
        if (this._differ) {
            const changes = this._differ.diff(this.infiniteForOf);
            if (changes) {
                this.applyChanges(changes);
            }
        }
    }

    private applyChanges(changes: IterableChanges<T>) {
        if (!this._collection) {
            this._collection = [];
        }
        let isMeasurementRequired = false;

        changes.forEachOperation((item: IterableChangeRecord<any>, adjustedPreviousIndex: number, currentIndex: number) => {
            if (item.previousIndex == null) {
                // new item
                isMeasurementRequired = true;
                this._collection.splice(currentIndex, 0, item.item);
            } else if (currentIndex == null) {
                // remove item
                isMeasurementRequired = true;
                this._collection.splice(adjustedPreviousIndex, 1);
            } else {
                // move item
                this._collection.splice(currentIndex, 0, this._collection.splice(adjustedPreviousIndex, 1)[0]);
            }
        });
        changes.forEachIdentityChange((record: any) => {
            this._collection[record.currentIndex] = record.item;
        });

        if (isMeasurementRequired) {
            this.requestMeasure();
        }

        this.requestLayout();
    }
}

Instead of directly react on data changes, we need to store the whole data set into a collection because we may use it later, But IterableDiffer is still a useful tool to manipulate our data collection without need to learn the actual data structure of source data. when any changes of data happen, we mark a isMeasurementRequired to determine whether we need to recalculate the list-holder height. This is needed when we change the size of collection. Whenever a change happen, a layout is requested.

Consider when we should do the layout. By learn the feature of our component, we can find three timing to do layout:

  • After a measurement. (data set size changes, container dimension changes)
  • scroll event happens.
  • data changes

The actual layout is the core process. we will break it into several parts.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
private layout() {
    if (this._isInLayout || !this._collection || this._collection.length === 0) {
        return;
    }
    this._isInLayout = true;
    let {width, height} = this._infiniteList.measure();
    this._containerWidth = width;
    this._containerHeight = height;
    this.findPositionInRange();
    for (let i = 0; i < this._viewContainerRef.length; i++) {
        let child = <EmbeddedViewRef<InfiniteRow>> this._viewContainerRef.get(i);
        this._viewContainerRef.detach(i);
        this._recycler.recycleView(child.context.index, child);
        i--;
    }
    this.insertViews();
    this._recycler.pruneScrapViews();
    this._isInLayout = false;
    this._invalidate = false;
}

we first check the prerequisites, we should not do layout when:

  • A layout is performing.
  • Collection is empty.

Then update containerHeight because we need this in findPositionInRange, After we do findPositionInRange( which is actually calculate the first and last index need to be in list-holder). we detach our existed views in list-holder. and re insert needed views back into it. after all job is done, we also prune the recycle bin to make sure it will not bloat too big.

How we implement findPositionInRange has been explained in the previous section. the start index and end index is exactly follow those formula to calculate. So the next important thing is adding views back to list-holder. the getView method first try to get views from recycle bin, if not find, create a new EmbeddedView with InfiniteRow object, this object is a data structure provide the data and some information like index, count. it has important property $implicit which is used by angular core to binding our data to view.

Once we get the view, dispatchLayout method will do the rest job to place views (applyStyle and insert).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
private insertViews() {
    for (let i = this._firstItemPosition; i <= this._lastItemPosition; i++) {
        let view = this.getView(i);
        this.dispatchLayout(i, view, false);
    }
}

private getView(position: number): ViewRef {
    let view = this._recycler.getView(position);
    let item = this._collection[position];
    let count = this._collection.length;
    if (!view) {
        view = this._template.createEmbeddedView(new InfiniteRow(item, position, count));
    } else {
        (view as EmbeddedViewRef<InfiniteRow>).context.$implicit = item;
        (view as EmbeddedViewRef<InfiniteRow>).context.index = position;
        (view as EmbeddedViewRef<InfiniteRow>).context.count = count;
    }
    return view;
}

private applyStyles(viewElement: HTMLElement, y: number) {
    viewElement.style.transform = `translate3d(0, ${y}px, 0)`;
    viewElement.style.webkitTransform = `translate3d(0, ${y}px, 0)`;
    viewElement.style.width = `${this._containerWidth}px`;
    viewElement.style.height = `${this._infiniteList.rowHeight}px`;
    viewElement.style.position = 'absolute';
}

private dispatchLayout(position: number, view: ViewRef, addBefore: boolean) {
    let startPosY = position * this._infiniteList.rowHeight;
    this.applyStyles((view as EmbeddedViewRef<InfiniteRow>).rootNodes[0], startPosY);
    if (addBefore) {
        this._viewContainerRef.insert(view, 0);
    } else {
        this._viewContainerRef.insert(view);
    }
    view.reattach();
}

We have nearly all done. But because we separate the component into two pieces. we haven’t implement the container component InfiniteList. It is responsible for listen to scroll event, resize event and set the height to list-holder, it has a simple dom structure:

1
2
3
4
5
6
7
<!-- infinite-list.html -->
<div class="infinite-list" #listContainer
     [ngClass]="scrollbarStyle">
    <div class="infinite-list-holder" [style.height]="holderHeightInPx">
        <ng-content></ng-content>
    </div>
</div>

By using ng-content we can put all element generated by InfiniteForOf directive. The important thing behind the html is a css definition. We must make infinite-list overflow-y: auto and give it a limit height (if you container element have a height, you can use 100%)

1
2
3
4
5
6
7
8
9
10
11
.infinite-list {
  overflow-y: auto;
  overflow-x: hidden;
  position: relative;
  contain: layout;          // this is performance trick, you can lookup this css property in MDN
  -webkit-overflow-scrolling: touch; // make a touch scroll has resilient.
  .infinite-list-holder {
    width: 100%;
    position: relative;     // relative is also a very important property. because of all its children will be position: absolute;
  }
}

In the InfiniteList class we heavily use the Observable from RxJS, if you are not familiar rxjs, you can find some material from the internet. There some important notice when implement this component.

  • Do all DOM event binding in AfterViewInit lifecycle hook. and besides, you should use setTimeout() to trigger a immediate measurement. Because this operation will change _containerWidth and _containerHeight in one tick which will cause an error in zone.js. so we need to schedule it to next tick.
  • scrollPosition subject is a BehaviorSubject, it has a convenience that a initial value will be emitted without need the scroll event. This is helpful to initialize the layout of InfiniteForOf.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
@Component({
    selector: 'infinite-list',
    templateUrl: 'infinite-list.html',
    styleUrls: ['infinite-list.less']
})
export class InfiniteList implements AfterViewInit, OnDestroy {
    private _holderHeight: number;
    private _containerWidth: number;
    private _containerHeight: number;

    private _subscription: Subscription = new Subscription();

    private _scrollPosition: BehaviorSubject<number> = new BehaviorSubject(0);
    private _sizeChange: BehaviorSubject<number[]> = new BehaviorSubject([0, 0]);

    @ViewChild('listContainer') listContainer: ElementRef;

    set holderHeight(height: number) {
        if (height) {
            this._holderHeight = height;
        }
    }

    get holderHeight(): number {
        return this._holderHeight;
    }

    get holderHeightInPx(): string {
        if (this.holderHeight) {
            return this.holderHeight + 'px';
        }
        return '100%';
    }

    /**
     * current scroll position.
     * @type {number}
     */
    get scrollPosition(): Observable<number> {
        return this._scrollPosition.asObservable();
    }

    /**
     * list container width and height.
     */
    get sizeChange(): Observable<number[]> {
        return this._sizeChange.asObservable();
    }

    @Input() rowHeight: number;

    ngAfterViewInit(): void {
        if (window) {
            this._subscription.add(Observable.fromEvent(window, 'resize')
                .subscribe(() => {
                    let {width, height} = this.measure();
                    this._sizeChange.next([width, height]);
                }));
        }
        this._subscription.add(Observable.fromEvent(this.listContainer.nativeElement, 'scroll')
            .map(() => {
                return this.listContainer.nativeElement.scrollTop;
            })
            .subscribe((scrollY: number) => {
                this._scrollPosition.next(scrollY);
            }));

        setTimeout(() => {
            let {width, height} = this.measure();
            this._sizeChange.next([width, height]);
        });
    }

    ngOnDestroy(): void {
        this._subscription.unsubscribe();
    }

    measure():{width: number, height: number} {
        if (this.listContainer && this.listContainer.nativeElement) {
            let rect = this.listContainer.nativeElement.getBoundingClientRect();
            this._containerWidth = rect.width - this.scrollbarWidth;
            this._containerHeight = rect.height;
            return {width: this._containerWidth, height: this._containerHeight};
        }
        return {width: 0, height: 0};
    }
}

By now, we have all done. let’s use our component and directive together to make a list with infinite smooth scroll.

1
2
3
4
5
6
7
<div class="demo-container" *ngIf="collection">
    <infinite-list [rowHeight]="140">
        <list-item-example *infiniteFor="let row of collection; let i = index" [item]="row" [index]="i">

        </list-item-example>
    </infinite-list>
</div>

In this case, we give the view height 140 to InfiniteList component and pass row local variable to ListItemExample component, the actual list item will be rendered by ListItemExample component.

The demo can be found at GitHub.

The real component has some additional feature which can tell the list item component current scroll state. This will be helpful to make some decision when user is performing a fast scroll and avoid unnecessary resource loading.