Epiforge.Extensions.Expressions 4.3.0

This library has useful tools for dealing with expressions:

• ExpressionEqualityComparer - Defines methods to support the comparison of expression trees for equality
• ExpressionExtensions, providing:
  • Duplicate - Duplicates the specified expression tree
  • SubstituteMethods - Recursively scans an expression tree to replace invocations of specific methods with replacement methods

Observable

This library accepts a LambdaExpression and arguments to pass to it, dissects the LambdaExpression's body, and hooks into change notification events for properties (INotifyPropertyChanged), collections (INotifyCollectionChanged), and dictionaries (Epiforge.Extensions.Collections.INotifyDictionaryChanged).

// Employee implements INotifyPropertyChanged
var elizabeth = Employee.GetByName("Elizabeth");
var observer = new ExpressionObserver();
var expr = observer.Observe(e => e.Name.Length, elizabeth);
// expr subscribed to elizabeth's PropertyChanged

Then, as changes involving any elements of the expression occur, a chain of automatic re-evaluation will get kicked off, possibly causing the observable expression's Evaluation property to change.

var elizabeth = Employee.GetByName("Elizabeth");
var observer = new ExpressionObserver();
var expr = observer.Observe(e => e.Name.Length, elizabeth);
// expr.Evaluation.Result == 9
elizabeth.Name = "Lizzy";
// expr.Evaluation.Result == 5

Also, since exceptions may be encountered after an observable expression was created due to subsequent element changes, observable expressions include a Fault property in their evaluations, which will be set to the exception that was encountered during evaluation.

var elizabeth = Employee.GetByName("Elizabeth");
var observer = new ExpressionObserver();
var expr = observer.Observe(e => e.Name.Length, elizabeth);
// expr.Evaluation.Fault is null
elizabeth.Name = null;
// expr.Evaluation.Fault is NullReferenceException

Observable expressions raise property change events of their own, so listen for those (kinda the whole point)!

var elizabeth = Employee.GetByName("Elizabeth");
var observer = new ExpressionObserver();
var expr = observer.Observe(e => e.Name.Length, elizabeth);
expr.PropertyChanged += (sender, e) =>
{
    if (e.PropertyName == "Evaluation")
    {
        var (fault, result) = expr.Evaluation;
        if (fault is not null)
        {
            // Whoops
        }
        else
        {
            // Do something with result
        }
    }
};

While an expression is working out its new value it can pass through results that were never simultaneously true of its inputs; an addition whose two operands both derive from the same property has to compute one of them before the other. You are not told about those. Every event you receive carries a value the expression genuinely held, so a subscriber that redraws or broadcasts on one does that work once rather than twice, the second time only to correct the first.

Nor are you told anything at all when a change leaves the value where it found it. That is decided by a comparison, using the same equality the expression uses everywhere else, and it happens before PropertyChanging rather than after — so a handler for that event still reads the previous value, and a pair of events always means the value really moved.

When you dispose of your observable expression, it will disconnect from all the events.

var elizabeth = Employee.GetByName("Elizabeth");
var observer = new ExpressionObserver();
using (var expr = observer.Observe(e => e.Name.Length, elizabeth))
{
    // expr subscribed to elizabeth's PropertyChanged
}
// expr unsubcribed from elizabeth's PropertyChanged

How an Expression Gets Observed

Observe takes a shortcut when it can and builds a graph when it cannot, deciding once when the observation is created. You receive the same values through the same events either way; the shortcut is just faster and lighter.

The shortcut handles an expression built from these:

• the argument, constants, and captured locals
• fields, on anything above — including static fields
• properties and indexers whose target is one of the above
• static properties
• operators, where one resolved to a method needs a return type nothing could dispose — == on strings qualifies
• method calls, on that same condition, where the target and every argument are themselves handled — string.IsNullOrEmpty(e.Name) qualifies

Everything else builds the graph: ?:, &&, || and ??; anything read through a property, such as e => e.Name.Length; a call to the get method of a property or an indexer, which is how an indexer written in C# arrives; object and collection construction; and anything you have configured the observer to ignore notifications for or to dispose.

To find out about a particular expression, ask:

var analysis = new DirectSubscriptionAnalyzer(options).Analyze(expression.Body);
// analysis.IsEligible is false
// analysis.Ineligibility is DirectSubscriptionIneligibility.DeferredBranch
// analysis.IneligibleExpression is the part responsible

Hand the analyzer the same options you hand the observer, since some of them decide what gets subscribed to at all. Set UseDirectSubscription to false if you would rather always have the graph; it is true by default.

Fields Are Read Once

Whatever a field held when an observation began is what that observation goes on using — a captured local, a field of your own class, and a static field alike. Assigning it afterward does not reach an observation that already exists. Static properties behave the same way, so e => e.Hired < DateTime.Now compares against the moment it was created for as long as it lives.

var threshold = low;
using var expr = observer.Observe(e => e.Salary > threshold.Amount, elizabeth);
threshold = high;    // expr is still comparing against low
low.Amount = 50000;  // expr re-evaluates
high.Amount = 90000; // expr does not

If you want the comparison to follow the value, do not assign the field — make the thing it points at a property of an object that notifies, and read that instead.

Observable expressions will also try to automatically dispose of disposable objects they create in the course of their evaluation when and where it makes sense. Use the ExpressionObserverOptions class for more direct control over this behavior. You can use the Optimizer property to specify an optimization method to invoke automatically during the observable expression creation process. We recommend Tuomas Hietanen's Linq.Expression.Optimizer, the utilization of which would look like so:

var options = new ExpressionObserverOptions { Optimizer = ExpressionOptimizer.tryVisit };

var a = Expression.Parameter(typeof(bool));
var b = Expression.Parameter(typeof(bool));

var lambda = Expression.Lambda<Func<bool, bool, bool>>
(
    Expression.AndAlso
    (
        Expression.Not(a),
        Expression.Not(b)
    ),
    a,
    b
); // lambda explicitly defined as (a, b) => !a && !b

var observer = new ExpressionObserver(options);
var expr = observer.Observe<bool>(lambda, false, false);
// optimizer has intervened and defined expr as (a, b) => !(a || b)
// (because Augustus De Morgan said they're essentially the same thing, but this involves less steps)

Observable Queries

This library provides re-implementations of LINQ operations, but instead of returning IEnumerable<T>s and simple values, these return IObservableCollectionQuery<T>s, IObservableDictionaryQuery<TKey, TValue>s, and IObservableScalarQuery<T>s. This is because, unlike traditional LINQ operations, these implementations continuously update their results until those results are disposed. What they hand back is a read-only view of the source: change the source, and the query brings itself up to date. Queries do not implement the mutating range collection and dictionary interfaces, because a query result is not somewhere you put things.

But... what could cause those updates?

• the source is enumerable, implements INotifyCollectionChanged, and raises a CollectionChanged event
• the source is a dictionary, implements Epiforge.Extensions.Collections.INotifyDictionaryChanged<TKey, TValue>, and raises a DictionaryChanged event
• the elements in the enumerable (or the values in the dictionary) implement INotifyPropertyChanged and raise a PropertyChanged event
• a reference enclosed by a selector or a predicate passed to the method implements INotifyCollectionChanged, Epiforge.Extensions.Collections.INotifyDictionaryChanged<TKey, TValue>, or INotifyPropertyChanged and raises one of their events

That last one might be a little surprising, but this is because all selectors and predicates passed to Observable Query methods become Observable Expressions (see above). This means that you will not be able to pass one that an ExpressionObserver cannot observe (e.g. a lambda expression that can't be converted to an expression tree or that contains nodes that are unsupported). But, in exchange for this, you get all kinds of notification plumbing that's just handled for you behind the scenes.

Suppose, for example, you're working on an app that displays a list of notes and you want the notes to be shown in descending order of when they were last edited.

var notes = new ObservableCollection<Note>();
var collectionObserver = new CollectionObserver();

var observedNotes = collectionObserver.ObserveReadOnlyList(notes);
var orderedNotes = observedNotes.ObserveOrderBy(note => note.LastEdited, isDescending: true);
notesViewControl.ItemsSource = orderedNotes;

From then on, as you add Notes to the notes observable collection, the IObservableCollectionQuery<Note> named orderedNotes will be kept ordered so that notesViewControl displays them in the preferred order.

Since IObservableCollectionQuery<T>'s are automatically subscribing to events for you, you do need to call Dispose on them when you don't need them any more.

void Page_Unload(object? sender, EventArgs e)
{
    orderedNotes.Dispose();
    observedNotes.Dispose();
}

Ahh, but what about exceptions? Well, Observable Expressions contain a Fault element in their Evaluation properties, but... you don't really see those Observable Expressions as an Observable Query caller, do ya? For that reason, Observable Queries all have OperationFault properties. You may subscribe to their PropertyChanging and PropertyChanged events to be notified when an Observable Expression or the overall Observable Query runs into a problem. If there is more than one fault in play, the value of OperationFault will be an AggregateException.

Dictionary queries adopt the key comparer of the dictionary they observe, discovering it through Epiforge.Extensions.Collections.Generic.IHashKeys<TKey> or a Dictionary<TKey, TValue>'s own Comparer, so a query over a case-insensitive dictionary is itself case-insensitive.

ObserveGroupBy, ObserveToLookup, and ObserveDistinct do not order their results the way LINQ does. Groupings are ordered by when they were created and the elements of a grouping by when they were added, rather than by where they occur in the source. This is deliberate: holding a grouping at the position of its key's first occurrence would mean moving that grouping every time an element was inserted ahead of it, announcing a change to something whose membership did not change, which is the opposite of what an Observable Query is for. Call ObserveOrderBy on the query, or on a grouping, when you want a defined order.

Reach for foreach rather than the indexer, because the difference between them is larger than it looks and grows with the collection. An enumeration takes the query's lock once and then walks a list, while the indexer takes that lock again for every element you ask for; on a large collection it must also find each one in a tree, because a query keeps its elements' positions in one so that a change repairs only what it touched. A query does remember the position it handed out last and searches outward from there, so asking for positions in order, or near one another, costs a fraction of asking for them at random, and what remains is mostly the repeated locking rather than the search. Walking ten thousand elements by index instead of by enumerator measured between thirty and fifty times slower in order, and around two hundred times out of order; at a hundred elements it was about fifteen, and there the repeated locking is the whole of it. Where you do need elements by position more than once, copy the query's contents and index the copy.

Since the ExpressionObserver has a number of options governing its behavior, you may optionally pass one you've made to the constructor of CollectionObserver to ensure those options are obeyed when Observable Expressions are created to enable your Observable Queries.

How Observable Queries Work and When to Use Them

It is worth being plain about what kind of thing this is, because "LINQ, but observable" undersells it and sets the wrong expectations.

A LINQ query is a description of a computation you run. Run it again and it does all of the work again. An Observable Query is not re-run. It is a small machine that holds the answer and repairs it, so when something changes, only the parts of the answer that depended on that thing are recomputed. The work is proportional to what changed rather than to how much data you have. If you want the name the literature uses for this idea, it is incremental, or self-adjusting, computation.

Three things that might otherwise look like arbitrary restrictions fall straight out of that:

1. Your selectors and predicates have to be expression trees rather than delegates because the machine has to read them to find out what they depend on. A delegate is opaque; there is nothing in it to subscribe to.
2. You have to dispose of a query because it is holding subscriptions to everything it depends on, and those subscriptions are the entire reason the answer stays right.
3. Faults reach you through OperationFault instead of being thrown, because the evaluation that failed happened later, on whatever thread raised the change. By then there is no call of yours left on the stack to throw out of.

What is not free is construction. Building the machine means building an observable expression for every element the query touches, and that is proportional to the size of the collection. So build a query once and hold onto it. Do not build one per frame, per request, or per keystroke. The bargain is that you pay up front and then stop paying to read.

Reading is also cheaper than being told. A query subscribes to the one it is built on only while something is subscribed to it, and a filtered query works out where a change landed, and describes it, only when something will receive that description. So subscribe when you need to be told what changed, and simply read the query when you only need its answer to be right.

Which is also how to decide whether you want one. If you compute a result once and move on, plain LINQ is cheaper and simpler, and you should use it. If a result has to stay correct across a long run of small changes, such as a list someone is looking at, a running total, or a filter someone is typing into, that is what these are for.