Linq Index (part1 introduction)

About two years ago i worked with some linq to object queries that processed a lot of data. There i thought about implementing some indexing strategy for Linq to objects. I tried some things but then i lost interest. About half a year ago i had the same problem again and so i started thinking about this indexing thing again. So i started to implement the LinqIndex project. You can download it from codeplex (LinqIndex) (the project contains the test i wrote to check the perfomance). In this and the next blog entries i will write about the problems i had while implementing it, how i solved some of this problems and i will show how to use LinqIndex.
For introduction i have to say that the LinqIndex project should not be used very carefully in productive code because there are many cases that are not covered by LinqIndex and the error handling is very poor.
So first of all i will show the structure of LinqIndex and what parts it contains.
The most important parts of LinqIndex are the IndexSpecification and the IndexDefinitionExtensions. This image shows the uml diagramm of this two classes.

With the IndexSpecification class it is possible to specifie an index. The IndexDefinitionExtension contains the Where, Join and GroupJoin extension methodes to use the spefied index columns. If you want to use LinqIndex that are the two classes you need. But to understand what LinqIndex does we need the classes from the LinqIndex.IndexBuilding namespace.

In the LinqIndex.IndexBuilding namespace the most importand classes are the EqualIndex, Index classes that represent the created index. In my next post i will show how you could use LinqIndex to speed up linq queries.

LinqIndex: (part1) (part2) ((part3)) (part4) (part5)


One Comment on “Linq Index (part1 introduction)”

  1. Elias Chatzigeorgiou says:

    Hi Netmatze,

    many thanks for the great article. I am currently working on a complex data manipulation
    project that reads from CSV files into in-memory LINQ collections and then performs
    complex queries upon them to calculate price based on specific rules.

    I tried using your library to improve LINQ queries, but seems like I am facing a bug.
    Specifically, I have build a multiple column index like the below:

    _IndexedDatamixCol = _datamixCol
    .CreateIndexKey(c => c.Bookable_from)
    .AddIndexKey(c => c.Bookable_to)
    .AddIndexKey(c => c.Min_stay)
    .AddIndexKey(c => c.Max_stay)
    .AddIndexKey(c => c.HotelID)
    .AddIndexKey(c => c.RoomType)
    .AddIndexKey(c => c.RoomTypeCode)
    .AddIndexKey(c => c.RoomViewCode)
    .AddIndexKey(c => c.RoomLocationCode)
    .AddIndexKey(c => c.Room_type_and_meal_code);

    but when I try to run some queries upon it, I am facing a null reference exception in
    the DoubleRightRotation / DoubleLeftRotation functions.

    I tried to patch them as below:

    public void DoubleRightRotation(FastTreeNode treeNode)
    FastTreeNode right = treeNode.RightTreeNode;
    FastTreeNode rightLeft = right.LeftTreeNode;
    FastTreeNode parent = treeNode.ParentTreeNode;
    //elias: was
    //FastTreeNode rightLeftLeft = rightLeft.LeftTreeNode;
    //FastTreeNode rightLeftRight = rightLeft.RightTreeNode;
    //elias: new
    FastTreeNode rightLeftLeft = rightLeft != null ? rightLeft.LeftTreeNode : null;
    FastTreeNode rightLeftRight = rightLeft != null ? rightLeft.RightTreeNode : null;

    but then I got further null references like:

    leftRight.ParentTreeNode = parent;

    where leftRight object is null.

    Any help would be appreciated. If you wish I can share the entire project and data
    files so you can check in detail.


If you have a note or a question please write a comment.

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.