Wednesday, November 9, 2011

LINQ To LDAP: Closing in on 2.0

It's been a while since I've posted. I moved across town, took a vacation, and have just been generally busy. But I'm back and with a few updates about 2.0.

Since the last time I posted, I've made a few API changes and added quite a bit more mapping support. One thing I want to talk about is some tweaks to paging. I've covered ToPage and PageAll in a previous post. PageAll, however, is being deprecated in version 2.0. In it's place you can do this:
//will page all results when enumerated based on the server max page size from LdapConfiguration
context.Query<User>();

//will create a page request for 10 results and stop there
context.Query<User>()
    .Take(10);

//will page all the results in groups of 50
context.Query<User>()
    .InPagesOf(50);

//will create a page request for 2 entries at a time up to a maximum of 10 results
context.Query<User>()
    .Take(10)
    .InPagesOf(2);

In cases when the take size is smaller than the page size, take size will be used.

This is part of the trunk, but is not in the 2.0 beta 2 release. I've updated most of the documentation so you can see a lot of the new stuff over at the codeplex page. I'll also be adding this project to Nuget soon.

Thursday, August 25, 2011

LINQ To LDAP: Hello DirectoryAttributes!

I had a very pleasant surprise when testing my DirectoryAttributes implementation. The biggest driving force for creating LINQ TO LDAP was simplifying the querying process. With dynamics, you had to create your filters manually which is nice for simple scenarios but can get messy pretty quickly.

Since I've gone back to a static type, I'm pleased to say this query works perfectly:
var query = context.Query(namingContext, objectClass: "Person")
    .Where(da => (Filter.Equal(da, "givenname", "Andrew") && Filter.Equal(da, "sn", "Fuller")) || 
        Filter.Equal(da, "cn", "Andrew Fuller"))
    .Select(da => new 
                     {
                          Guid = da.GetGuid("objectguid"), 
                          EmployeeId = da.GetInt("employeeid")
                     });

var user = query.FirstOrDefault();

//Produces this filter:
(&(objectClass=Person)(|(&(givenname=Andrew)(sn=Fuller))(cn=Andrew Fuller)))
Attributes: objectguid, employeeid
Projections are supported as well. This will look at the parameters for the Get methods so your anonymous object properties can be called anything.

The Filter class still only supports Equal and Approximately so I'll need to fill it out with the other operations. Happy coding!

Wednesday, August 24, 2011

LINQ To LDAP: Bye Bye Dynamics

Well, it was a fun ride while it lasted. Given the performance penalty for blindly accessing byte arrays and the guessing game that I have to handle to avoid it, it only makes sense to peel away the abstraction and give control back to the user.

So in place of dynamics I created IDirectoryAttributes and DirectoryAttributes. If you've ever worked with IDataReader, it will feel very familiar. Here are the get methods for the interface:
string DistinguishedName { get; }

object GetValue(string attribute);

DirectoryAttribute Get(string attribute);

byte[] GetBytes(string attribute);

string[] GetStrings(string attribute);

string GetString(string attribute);

byte? GetByte(string attribute);

int? GetInt(string attribute);

long? GetLong(string attribute);

double? GetDouble(string attribute);

decimal? GetDecimal(string attribute);

short? GetShort(string attribute);

float? GetFloat(string attribute);

bool? GetBoolean(string attribute);

DateTime? GetDateTime(string attribute, string format = "yyyyMMddHHmmss.0Z");

Guid? GetGuid(string attribute);

SecurityIdentifier GetSecurityIdentifier(string attribute);

IEnumerable<byte[]> GetByteArrays(string attribute);

Pretty standard. Attribute is case insensitive. If it's not found then null will be returned. Since everything is either a string or byte array from the directory, if it fails to convert then a FormatException will be thrown. GetValue will try to guess the type like before and is only here to support the implementation of IEnumerable<string, object>. Of course if there's a datatype here that's missing, just use Get and you'll get direct access to the DirectoryAttribute.

So what about change tracking? Since DirectoryAttributes is just a wrapper for SearchResultEntry, setting values is tracked in an internal dictionary. Just call SetValue(attributeName, value) and it will take care of the rest.
var factory = new LdapConnectionFactory("localhost");
string namingContext = "CN=Users,CN=Employees,DC=Northwind,DC=local";

using (IDirectoryContext context = new DirectoryContext(factory.GetConnection(), true))
{
    var query = context.Query(namingContext, objectClass: "Person")
        .Where("cn=Andrew Fuller")
        .Select("cn");

    var user = query.FirstOrDefault();

    user.SetValue("employeeid", 1);

    var updated = context.Update(user);

    Console.WriteLine(updated.GetInt("employeeid"));
    Console.WriteLine(updated.GetGuid("objectguid"));
}

That's it. I hope this helps with performance and satisfies anyone who can't / won't map classes.

Sunday, August 14, 2011

LINQ To LDAP: CR(U)D

So now that I've covered Adding entries, let's talk about updating. Updating your entries is a little different than in a RDBMS.

Using S.DS.P directly you can make modifications to an entry like so:
var firstNameMod = new DirectoryAttributeModification
                        {
                            Name = "givenname",
                            Operation = DirectoryAttributeOperation.Replace,

                        };
firstNameMod.Add("Jack");

var commentMod = new DirectoryAttributeModification
                        {
                            Name = "comment",
                            Operation = DirectoryAttributeOperation.Add,

                        };
commentMod.Add("add a property to this entry");

var dateOfBirthMod = new DirectoryAttributeModification
                        {
                            Name = "dateofbirth",
                            Operation = DirectoryAttributeOperation.Delete,

                        };

ModifyRequest request = new ModifyRequest("CN=John Doe,CN=Users,CN=Employees,DC=Northwind,DC=local", new [] {firstNameMod, commentMod, dateOfBirthMod });

ldapConnection.SendRequest(request);

So what you see here is a request to modify John Doe by changing his given name to Jack, adding a brand new attribute called comment, and deleting his date of birth. Those last two modification will actually make a change to the structure of the object. So how do you do this with LINQ To LDAP? Replace (updating) operations are pretty easy, but I separate adding or removing attributes into different methods because it's difficult to identify the intention (i.e. are you trying to delete an attribute because you set it to null).

I'll continue with my previous User class example, but with a few modifications for change tracking. The primary change is sub typing DirectoryObjectBase (built into LINQ to LDAP) and changing your setters to look like this:
set
{
     _firstName = value;
     AttributeChanged("FirstName");
}

Similar to INotifyPropertyChanged, just call the AttributeChanged when the property changes. However, I don't use the interface because I didn't want to worry about events subscriptions. But don't feel like you have to use DirectoryObjectBase. LINQ To LDAP will work just as well without it.

Change tracking on entries is enabled when you use either a full projection ( Select(u => u) ), no projection or GetByDN. Since I support projections of mapped entries, I had to disable it so you wouldn't accidentally update attributes to an empty value. If your object sub types DirectoryObjectBase and you try to update it without change tracking being enabled, then I actually throw an exception. For dynamic queries, change tracking is already built in so you don't have to do anything special.

So let's update!
var factory = new LdapConnectionFactory("localhost");

//mapped
using (var context = new DirectoryContext(factory.GetConnection(), true))
{
    var user = context.GetByDN<User>("CN=John Doe,CN=Users,CN=Employees,DC=Northwind,DC=local");
    user.FirstName = "Jack";
    context.Update(user);
}

//dynamic
using (var context = new DirectoryContext(factory.GetConnection(), true))
{
    dynamic user = context.GetByDN("CN=John Doe,CN=Users,CN=Employees,DC=Northwind,DC=local", "givenname");
    user.givenname = "Jack";
    context.Update(user.DistinguishedName, user);
}

//update structure
using (var context = new DirectoryContext(factory.GetConnection(), true))
{
    context.AddAttribute(
        "CN=John Doe,CN=Users,CN=Employees,DC=Northwind,DC=local", "comment", 
        "add a property to this entry");

    context.DeleteAttribute(
        "CN=John Doe,CN=Users,CN=Employees,DC=Northwind,DC=local", 
        "dateofbirth");
}

There's one more operation for updating. In order to change an entry's distinguished name or move it to a different location you have to issue a ModifyDN request. This kind of request looks like this:
//Move entry using raw S.DS.P
var dnRequest = new ModifyDNRequest
{
	DistinguishedName = "CN=John Doe,CN=Users,CN=Employees,DC=Northwind,DC=local"
	NewParentDistinguishedName = "CN=Deactivated Users,CN=Employees,DC=Northwind,DC=local",
	NewName = "CN=John Doe"
};

connection.SendRequest(dnRequest);

//Move entry using LINQ To LDAP
string newDn = directoryContext.MoveEntry(
	"CN=John Doe,CN=Users,CN=Employees,DC=Northwind,DC=local", 
	"CN=Deactivated Users,CN=Employees,DC=Northwind,DC=local");


//Rename entry using raw S.DS.P
var dnRequest = new ModifyDNRequest
{
	DistinguishedName = "CN=John Doe,CN=Users,CN=Employees,DC=Northwind,DC=local"
	NewParentDistinguishedName = "CN=Users,CN=Employees,DC=Northwind,DC=local",
	NewName = "CN=Jack Doe"
};

connection.SendRequest(dnRequest);

//Rename entry using LINQ To LDAP
string newDn = directoryContext.RenameEntry(
	"CN=John Doe,CN=Users,CN=Employees,DC=Northwind,DC=local", 
	"Jack Doe");

And that's all there is to it. One thing that you must know is there's no concept of transactions in LDAP (at least not yet) so if you want to update multiple entries, be aware that if it fails half-way through, there's no rollback.

LINQ To LDAP 2.0 Beta is out over at CodePlex!

Sunday, July 31, 2011

LINQ To LDAP: GetValues Is Your Friend

I was finally able to put some time towards tackling a strange bug in LINQ to LDAP. Given this directory structure:
  • Company\Sites\DIV_A\Accounts\users\Users 1       -> 387 users
  • Company\Sites\DIV_A\Accounts\users\Users 2       -> 303 users
  • Company\Sites\DIV_A\Accounts\users\...                -> 21 users
  • Company\Sites\DIV_A\Accounts\users           Total -> 711 users

This query would perform horribly (as long as 20 seconds) when querying for all entries:
string namingContext = "OU=Users,OU=Accounts,OU=DIV_A,OU=Sites,DC=company,DC=local";
var example = new
{
 DistinguishedName = "",
 Name = "",
 Cn = "",
 ObjectGuid = default(Guid),
 ObjectSid = default(SecurityIdentifier)
};

var query = context.Query(example, SearchScope.Subtree, namingContext, objectClass: "Person")
  .Select(u => u);

var users = query.PageAll(10000);

If I remove ObjectGuid and ObjectSid then performance goes back to normal. This led me to the code that accesses the attributes. I was using the default index accessor when I should have been using GetValues for multi-valued attributes:
SearchResultAttributeCollection collection = //code to get collection from entry
//bad for multi-valued attributes
object guid = collection["objectguid"][0];

//much better
object guid = collection["objectguid"].GetValues(typeof(byte[]))[0];

So what gives? Why does GetValues perform so much better? It's recommended in the Introduction to System.DirectoryServices.Protocols, but other than that the Internet is pretty quiet on the subject. I fired up reflector and it looks like it tries to parse the bytes to a string, which fails so it catches the exception and returns the bytes as is. The performance issue is being caused by an exception being thrown for every byte array that tries to be retrieved...ouch.

The way mapping is done in LINQ to LDAP makes it easy to fix this, but dynamic queries are a little trickier. The duct tape fix is to hard code a list of popular guid / sid names and check for those:
public class SearchResultAttributeDictionary : Dictionary<string, object="">
{
    private const string ObjectGuid = "objectguid";
    private const string ObjectSid = "objectsid";

    private static readonly Dictionary<string, string> ByteProperties =
        new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
            {
                {"objectguid", "objectguid"},
                {"objectsid", "objectsid"},
                {"Ibm-entryuuid", "Ibm-entryuuid"},
                {"guID", "guID"},
                {"Orclguid", "Orclguid"},
                {"Nsuniqueid", "Nsuniqueid"}
            };

    public SearchResultAttributeDictionary(SearchResultAttributeCollection collection)
        : base(StringComparer.OrdinalIgnoreCase)
    {
        if (collection == null) return;
        foreach (var attribute in collection.AttributeNames.Cast<string>())
        {
            var value = collection[attribute];
            if (value.Count == 1)
            {
                Add(attribute,
                    ByteProperties.ContainsKey(attribute) 
                        ? value.GetValues(typeof (byte[]))[0] 
                        : value[0]);
            }
            else if (value.Count > 1)
            {
                var type = value[0].GetType();
                Add(attribute, value.GetValues(type));
            }
            else
            {
                Add(attribute, null);
            }
        }
    }
}

This is the best solution I could come up with for now. If someone has a better one I'm open to suggestions.

Wednesday, July 27, 2011

LINQ To LDAP: (C)RUD

Just like any other data store you can create, update, and delete data in a directory.

Here's how you create entries using S.DS.P:

LdapConnection connection = new LdapConnection("localhost");

string distinguishedName = "CN=John Doe,CN=Users,CN=Employees,DC=Northwind,DC=local";
AddRequest request = new AddRequest(distinguishedName, "User");

request.Attributes.Add(new DirectoryAttribute("givenname", "John"));
request.Attributes.Add(new DirectoryAttribute("sn", "Doe"));
request.Attributes.Add(new DirectoryAttribute("employeeid", "1"));

connection.SendRequest(request);

Looks pretty straightforward. You give your entry a primary key (distinguished name), an object class, and then populate the attributes for the new entry and submit it to the directory.

So here's how you do it with mapped classes:
public abstract class DirectoryObject
{
    [DistinguishedName]
    public string DistinguishedName { get; set; }

    [DirectoryAttribute(StoreGenerated = true)]
    public DateTime? WhenChanged { get; set; }

    [DirectoryAttribute("cn")]
    public string CommonName { get; set; }

    [DirectoryAttribute(StoreGenerated = true)]
    public DateTime? WhenCreated { get; set; }

    [DirectoryAttribute("objectguid", StoreGenerated = true)]
    public Guid Guid { get; set; }

    [DirectoryAttribute]
    public string Name { get; set; }
}

[DirectorySchema(NamingContext, ObjectCategory = "Person", ObjectClass = "user")]
public class User : DirectoryObject
{
    private const string NamingContext = "CN=Users 1,CN=TestContainer,CN=Employees,DC=Northwind,DC=local";

    [DirectoryAttribute("objectsid", StoreGenerated = true)]
    public SecurityIdentifier SID { get; set; }

    [DirectoryAttribute("givenname")]
    public string FirstName { get; set; }

    [DirectoryAttribute("sn")]
    public string LastName { get; set; }

    [DirectoryAttribute("directreports")]
    public string[] Employees { get; set; }

    [DirectoryAttribute]
    public string Title { get; set; }

    [DirectoryAttribute]
    public string PostalCode { get; set; }

    [DirectoryAttribute(ImageFormat = ImageType.Png)]
    public Bitmap Photo { get; set; }

    [DirectoryAttribute("l")]
    public string City { get; set; }

    [DirectoryAttribute("c")]
    public string Country { get; set; }

    [DirectoryAttribute]
    public string EmployeeID { get; set; }

    [DirectoryAttribute]
    public string TelephoneNumber { get; set; }

    [DirectoryAttribute("pwdlastset", DateTimeFormat = null, StoreGenerated = true)]
    public DateTime? PasswordLastSet { get; set; }

    [DirectoryAttribute]
    public string Street { get; set; }

    public void SetDistinguishedName()
    {
        DistinguishedName = "CN=" + CommonName + "," + NamingContext;
    }
}

There are a few items to note here. First is the StoreGenerated property of DirectoryAttribute. This allows the DirectoryContext to only update attributes that the store doesn't manage. The second is the DistinguishedName attribute which mainly helps when calling Add and Update. The third is the DirectoryObject which isn't a part of LINQ to LDAP, but is an example abstract base class that all directory objects can sub-type.
User user = new User
                {
                    City = "Some City",
                    CommonName = "John Doe",
                    Country = "US",
                    EmployeeID = "1",
                    FirstName = "John",
                    LastName = "Doe",
                    Name = "Doe, John",
                    Street = "1234 Street",
                    Title = "Unknown",
                    PostalCode = "12345",
                    TelephoneNumber = "123-456-7890"
                };

user.SetDistinguishedName();
var factory = new LdapConnectionFactory("localhost");
using (var context = new DirectoryContext(factory.GetConnection(), true))
{
    User added = context.Add(user);
}

I create a new user and initialize its properties. Whenever an object is added or updated, a fresh version is retrieved using GetByDN.

Alternatively, you can perform the same operation using a dictionary:
string dn = "CN=John Doe,CN=Users 1,CN=TestContainer,CN=Employees,DC=Northwind,DC=local";
var attributes = new Dictionary<string, object>
                        {
                            {"l", "Some City"},
                            {"cn", "John Doe"},
                            {"c", "US"},
                            {"EmployeeID", "1"},
                            {"givenname", "John"},
                            {"sn", "Doe"},
                            {"Name", "Doe, John"},
                            {"Street", "1234 Street"},
                            {"Title", "Unknown"},
                            {"PostalCode", "12345"},
                            {"TelephoneNumber", "123-456-7890"}
                        };

var factory = new LdapConnectionFactory("localhost");
using (var context = new DirectoryContext(factory.GetConnection(), true))
{
    IDictionary<string, object> added = context.Add(dn, "User", attributes);
}

Or by converting an anonymous object to a dictionary:
string dn = "CN=John Doe,CN=Users 1,CN=TestContainer,CN=Employees,DC=Northwind,DC=local";
var user = new
                {
                    City = "Some City",
                    CommonName = "John Doe",
                    Country = "US",
                    EmployeeID = "1",
                    FirstName = "John",
                    LastName = "Doe",
                    Name = "Doe, John",
                    Street = "1234 Street",
                    Title = "Unknown",
                    PostalCode = "12345",
                    TelephoneNumber = "123-456-7890"
                };

var factory = new LdapConnectionFactory("localhost");
using (var context = new DirectoryContext(factory.GetConnection(), true))
{
    IDictionary<string, object> added = context.Add(dn, "User", user.ToDictionary());
}

ToDictionary is just an extension method that reflects over an attribute and creates a dictionary from its properties.

So I think that covers how to add new entries. Questions, thoughts, improvements?

Sunday, July 10, 2011

Bitmap Mapping

Between a busy job, house hunting and a much needed vacation I've fallen out of the habit of blogging. My first post back I want to talk about an undocumented feature for LINQ to LDAP and why I'm thinking of removing it.

Currently LINQ to LDAP supports mapping images stored in a directory as Bitmaps. I thought this was cool since all you had to do was map it with an image type and it just worked (updates as well in the new 2.0 changes). However, I've left it undocumented since Bitmaps use GDI+ and if you're not careful you can introduce a memory leak. Imagine blindly querying for all users in your directory and suddenly there's 10,000 (assuming you're in a large organization or university) bitmaps being loaded up.

So if this feature gets removed here is how you could still support images:

[DirectorySchema("OU=Users,DC=mycompany,DC=com", ObjectClass = "User")]
public class User : IDisposable
{
     [DirectoryAttribute("Photo")]
     public byte[] PhotoBytes { get; set; }
     
     private Bitmap _photo;
     public Bitmap Photo
     {
          get
          {
               if (PhotoBytes == null) return null;
               
               if (_photo == null)
               {
                    _photo = new Bitmap(new MemoryStream(PhotoBytes));
               }
               return _photo;
          }
          set
          {
               if (_photo != null) _photo.Dispose();
               if (value == null)
               {
                    _photo = null;
                    PhotoBytes = null;
               }
               else
               {
                    _photo = value;
                    using (var stream = new MemoryStream())
                    {
                         _photo.Save(stream, ImageFormat.Png); //whatever format
                         PhotoBytes = stream.ToArray();
                    }
               }
          }
     }

     public void Dispose()
     {
          if (_photo != null) 
          {
               _photo.Dispose();
               _photo = null;
          }
     }
}

So is this even a big deal? Will anyone care if I removed bitmap mapping?