LINQ To LDAP: Sorting

I recently checked in my implementation of sorting for LINQ to LDAP. It's a pretty straight forward implementation. I have sort options that I pick up when visiting the expression and they get applied when I execute the request. Here's my QueryCommand that handles standard execution:

internal class StandardQueryCommand : QueryCommand
{
    public StandardQueryCommand(IQueryCommandOptions options, IObjectMapping mapping)
        : base(options, mapping, true)
    {
    }

    public override object Execute(DirectoryConnection connection, SearchScope scope)
    {
        SearchRequest.Scope = scope;
            
        var isSorted = Options.SortingOptions != null;
        if (isSorted)
        {
            var sortRequest = new SortRequestControl(Options.SortingOptions.Keys)
                                    {
                                        IsCritical = false
                                    };
            SearchRequest.Controls.Add(sortRequest);
        }
            
        return Options.PagingOptions == null 
            ? HandleStandardRequest(isSorted, connection) 
            : HandlePagedRequest(connection);
    }

    private object HandleStandardRequest(bool isSorted, DirectoryConnection connection)
    {
        if (isSorted)
        {
            var pageRequest = new PageResultRequestControl(LdapConfiguration.ServerMaxPageSize);
            SearchRequest.Controls.Add(pageRequest);
        }
        var response = connection.SendRequest(SearchRequest) as SearchResponse;

        AssertSuccess(response);

        AssertSortSuccess(response.Controls);
        return Options.GetEnumerator(response.Entries);
    }

    private object HandlePagedRequest(DirectoryConnection connection)
    {
        var pageRequestControl = new PageResultRequestControl(Options.PagingOptions.PageSize)
                                        {
                                            Cookie = Options.PagingOptions.NextPage
                                        };

        SearchRequest.Controls.Add(pageRequestControl);

        var response = connection.SendRequest(SearchRequest) as SearchResponse;

        AssertSuccess(response);

        AssertSortSuccess(response.Controls);
        var pageResponseControl = GetControl<PageResultResponseControl>(response.Controls);
        var parameters = new[]
                                {
                                    Options.PagingOptions.PageSize, 
                                    pageResponseControl.Cookie,
                                    Options.GetEnumerator(response.Entries)
                                };

        return Activator.CreateInstance(
            typeof(LdapPage<>).MakeGenericType(Options.GetEnumeratorReturnType()),
            parameters);
    }

    private static void AssertSortSuccess(IEnumerable<DirectoryControl> controls)
    {
        var control = GetControl<SortResponseControl>(controls);

        if (control != null && control.Result != ResultCode.Success)
        {
            throw new LdapException(
                string.Format("Sort request returned {0}", control.Result));
        }
    }
}

Sort requests require paging or else you'll get a size limit exceeded if you have too many results. I thought this was all I needed to worry about, but when I tried to filter 50K entries I found another gem. The response of the sort request was "UnwillingToPerform". I felt like my laptop just told me to take a hike. I Googled and only turned up that something was wrong with my sort. I looked at my query and it hit me:

var orderBy = context.Query<User>()
    .OrderByDescending(u => u.LastName)
    .Select(u => new User {CommonName = u.CommonName, FirstName = u.FirstName, LastName = u.LastName})
    .ToPage(50);

Basically this is the same error as size limit exceeded. Even with a page request, the filter is too broad to effectively perform a server side sort, so it fails. I applied a filter and everything worked.

Sorting is not an efficient operation for the server which is why it has a size limit. Unless you're working on a horribly constrained client, I recommend doing your sorting there.

LINQ To LDAP: Paging

Let's talk about paging. The first thing you have to understand is paging with LDAP is done very differently than paging with RDBMS. Typically you'd have something like this:

var paged = context.Users.Skip(0).Take(10)

where the skip amount is the index of the first result and the take amount is the page size. LINQ to Objects works with this as well as most LINQ to SQL implementations. This supports forwards and backwards paging pretty easily since you can always control the position from which you start paging.

LDAP, however, treats paging a little differently. Here's an example:

private byte[] _nextPage;
private int _pageSize = 2;
private void Page()
{    
    var control = new PageResultRequestControl(_pageSize) {Cookie = _nextPage};

    var request = new SearchRequest
                        {
                            DistinguishedName = "CN=Users,CN=Employees,DC=Northwind,DC=local",
                            Filter = "objectClass=user",
                            Scope = SearchScope.Subtree
                        };
    request.Controls.Add(control);
    using (var connection = new LdapConnection("localhost"))
    {
        var response = connection.SendRequest(request) as SearchResponse;
        var next = response.Controls[0] as PageResultResponseControl;
        _nextPage = next.Cookie;
    }
}

You set the page size and you use a paging control to let the server know the position of the page. The server will generate a new cookie that will give you the next page. From the RFC:

The client MUST consider the cookie to be an opaque structure and make no assumptions about its internal organization or value. When the client wants to retrieve more entries for the result set, it MUST send to the server a searchRequest with all values identical to the initial request with the exception of the messageID, the cookie, and optionally a modified pageSize. The cookie MUST be the octet string on the last searchResultDone response returned by the server. Returning cookies from previous searchResultDone responses besides the last one is undefined, as the server implementation may restrict cookies from being reused.

If you know that the LDAP server you're using will work with backwards paging, you can implement client side tracking of the previous cookies, but I can't support it since it's unpredictable between servers. Either way, here's how I added paging support.

So let's start with how to page:

//start paging
var oldPage = context.Query<User>()
   .Where(u => u.FirstName.StartsWith("A"))
   .Select(u => u.DistinguishedName)
   .ToPage(10);

//go to next page
if (oldPage.HasNextPage)
{
   var nextPage = context.Query<User>()
      .Where(u => u.FirstName.StartsWith("A"))
      .Select(u => u.DistinguishedName)
      .ToPage(oldPage.PageSize, oldPage.NextPage);
}

ToPage is an extension method that returns a custom collection I created to allow you to get some basic information about the page and easily move to the next page. Here's PagedCollection:

public interface IPagedCollection<out T> : IEnumerable<T>
{
    int Count { get; }
    int PageSize { get; }
    bool HasNextPage { get; }
    byte[] NextPage { get; }
}

public class PagedCollection<T> : ReadOnlyCollection<T>, IPagedCollection<T>
{
    public PagedCollection(int pageSize, byte[] nextPage, object enumerator)
        : this(pageSize, nextPage, GetList((IEnumerator<T>)enumerator))
    {
    }

    public PagedCollection(int pageSize, byte[] nextPage, IList<T> page)
        : base(page)
    {
        PageSize = pageSize;
        NextPage = nextPage;

        HasNextPage = NextPage != null && NextPage.Length > 0;
    }

    private static IList<T> GetList(IEnumerator<T> enumerator)
    {
        var list = new List<T>();

        while (enumerator.MoveNext()) 
            list.Add(enumerator.Current);

        return list;
    }

    public int PageSize { get; private set; }
    public bool HasNextPage { get; private set; }
    public byte[] NextPage { get; private set; }
}

I've committed these changes to the trunk and updated the sample project. Go ahead and try it out. Let me know if there's anything I can add / change.

LINQ To LDAP: Version 0.9

You can head over to codeplex and download a beta version of LINQ to LDAP. I've tested it, but I didn't feel comfortable making it a 1.0 release without getting feedback. I also updated the documentation over there as well.

Download it now!

LINQ To LDAP: String to Int follow-up

I said I would clean up my DelegateBuilder class when I could. Here's the updated version that now even supports enum mapping. A lot cleaner in my opinion.

private static readonly Func<object, Type, object> ConvertStringToValueTypeIfNecessaryFunction =
    (o, t) =>
        {
            string str;
            if ((str = o as string) != null && t.IsValueType)
            {
                if (t.IsGenericType && string.Empty.Equals(str))
                {
                    return null;
                }

                return t.IsEnum 
                    ? Enum.Parse(t, str, true) 
                    : Convert.ChangeType(str, t);
            }
            return o;
        };