Directory Sync

Finding Directory Changes with LDAP

Posted by Michael Ryan on April 30, 2019
We've all done our fair share of LDAP operations, but one of the rarely used and not well understood aspects of this is the LDAP Control.

There are a variety of controls, the one I'm looking at specifically today is LDAP_SERVER_DIRSYNC_OID (1.2.840.113556.1.4.841). This control for ldap searching revolves around sending you results that include only objects that changed since your last search.

The idea here is you can send a search request now, get your results (plus a cookie) then an hour from now send that same cookie and it will only respond with results where the values have changed in the meanwhile.

There's a flag you can add to this to see only incremental changes for forward linked attributes (like member of a group).

So let's get right into how to do this.

The Powershell .NET Way

The first way we'll look at how to do this is with .NET in Powershell since it is the simplest.

Firstly we need to instantiate a couple variables. One for where we want to store our cookie once we get it, and one with the credentials we're going to use to bind to the directory:
$cookie="C:\dev\cookie"
$cred=get-credential

This next part you can all do as part of the same command like this:
$search=new-object System.DirectoryServices.DirectorySearcher((new-object System.DirectoryServices.DirectoryEntry("LDAP://domaincontroller01.contoso.com:389/DC=contoso,DC=com", $cred.username, $cred.GetNetworkCredential().password)), "(objectclass=group)", @("member"))

or break it down into its elements to understand it better and make it a bit more readable
#The path to our ldap server
$ldap_path="LDAP://domaincontroller01.contoso.com:389/DC=contoso,DC=com"

#Our filter
$filter="(objectclass=group)"

#The properties we want - This is important since these are the properties we'll be watching for changes
$properties=@("member")

#Now we're into the meat - we create a .NET DirectoryEntry with our provided username and password
$entry=new-object System.DirectoryServices.DirectoryEntry($ldap_path, $cred.username, $cred.GetNetworkCredential().password)

#Then we declare a new directorysearcher from our entry and give it our filter and properties
$searcher=new-object System.DirectoryServices.DirectorySearcher($entry, $filter, $properties)

Now some of you may have even created this kind of DirectorySearcher before, and know that now we're ready to just run $searcher.findall() and it will show us all our group objects and their members.

But in this case before we do that we want to declare the DIRSYNC control and assign it to the searcher and include our previous cookie if we had one.
#Create a DIRSYNC control object
$sync_control=new-object System.DirectoryServices.DirectorySynchronization

#And assign the IncrementalValues flag to it. This makes it so that multivalue attributes (like member) will only show us changes instead of all the values
$sync_control.Option=[DirectoryServices.Protocols.DirectorySynchronizationOptions]::IncrementalValues

#Check if we have a cookie written, if we do - assign it
if(Test-Path $cookie) {$sync_control.ResetDirectorySynchronizationCookie(([System.IO.File]::ReadAllBytes($cookie)))}

Now the first time you run this it won't look much different than your standard search, since everything that changed since our null cookie is...well, everything. Th interesting part of this comes when we look at our results and store the cookie away to search later.
#Grab our changes - which this time we don't really care that much about
$changes=$searcher.findAll()

#Get the cookie from our response
$thinmint = $sync_control.GetDirectorySynchronizationCookie()

#And store it away in our file path
[System.IO.File]::WriteAllBytes($cookie, $thinmint)

Now that we have the cookie stored, all we have to do is run it all again, and this time $changes will only contain entries that have had their member attribute changed since we last looked!

Reading Results

I know what you might be thinking, "But Mike, these results are pretty weird."

For responses for normal attributes that aren't forwardlinked like member is, the responses are way easier to read. For example, if we looked at user objects with useraccountcontrol, our results would just be the new useraccountcontrol value for any user which had that value change since our last cookie. Simple enough.

But with the member attribute it could change to add or remove members, and we want to be aware of both, so it send us a couple properties we can look at. Lets take a look at one of our response objects now:
PS C:\WINDOWS\system32> $changes[0].Properties
Name Value
---- -----
member;range=0-0 {CN=BobSmith,OU=Users,DC=contoso,DC=com, CN=Voldemort,OU=Users,DC=contoso,DC=com}
member;range=1-1 {CN=MikeRyan,OU=Users,DC=contoso,DC=com}
objectguid {125 143 79 49 216 198 174 71 139 96 44 94 235 49 233 230}
adspath {LDAP://domaincontroller01.contoso.com:389/CN=CoolGuys,OU=Groups,DC=contoso,DC=com}

member;range=0-0 is its way of telling us "these members were removed". So we can see that both Voldemort and BobSmith stopped being cool guys. As you probably guessed, member;range=1-1 is the people who got added, so MikeRyan started being a cool guy. Remember these are only the changes that happened since our last cookie, so we know these were pretty recent.

If you needed to know the exact time someone was added or removed you could always look at Replication Metadata to find out.

So now we're at a point where if we just ran the script in it's entirity the new cookie is already stored, and we can run it again to see more changes!

The Python ldap3 Way

Since I've been using python so much for my ldap operations I thought I would look into how to do this the python way. It feels a little more complex, but here's what I came up with.

This part is pretty standard of creating a new ldap connection in python using ldap3
#Just like we did before - instantiate some credentials and a cookie path
import ldap3, getpass
cookiepath="C:\\dev\\cookie"
acctname="contoso\\MikeRyan-Admin"
password=getpass.getpass()

server = ldap3.Server(
  host="domaincontroller01.contoso.com",
  get_info = ldap3.ALL,
  port= 389
)

conn = ldap3.Connection(
  server=server,
  user=acctname,
  password=password,
  auto_bind = True
)

default=server.info.raw['defaultNamingContext'][0].decode()

And from here we could just do standard searches using the conn.search() method.

But we want to send the directorysync control. In python I ended up having to construct the "Sequence" object to pass to the control.
I got some info on how to do that from this Microsoft Documentation and this RFC Documentation.

Here's the code I used to build the control out:
#Import some structures from the python ASN1 implementation
from pyasn1.type.univ import Sequence, Integer, Boolean, OctetString
from pyasn1.type.namedtype import NamedTypes, NamedType
#Define a Sequence class with the different
from pyasn1.type.univ import Sequence, Integer, Boolean, OctetString from pyasn1.type.namedtype import NamedTypes, NamedType class dirsync_control_builder(Sequence):
  componentType = NamedTypes(
    NamedType('Flags', Integer()),
    NamedType('maxAttributeCount', Integer()),
    NamedType('Cookie', OctetString()),
  )


#Then we go in and define the properties of our new sequence object
dirsync = dirsync_control_builder()
dirsync['Flags'] = 0
dirsync['maxAttributeCount'] = 1000
try:
  with open(cookie,"rb") as f:
    dirsync['Cookie']=f.read()
except FileNotFoundError:
  dirsync['Cookie'] = ''

dirsync_control = ldap3.protocol.controls.build_control('1.2.840.113556.1.4.841', True, dirsync) conn.search(   default,
  '(objectclass=user)',
  paged_size = 1000,
  attributes=['useraccountcontrol'],
  controls=(dirsync_control,)
)

If we look at our results here it's a little different than we got in powershell, before providing the cookie in powershell it iterated through all the results for us. In this case we've got a response including a "more_results" value, which we can see if we look at conn.result

So we need to run through a loop until there are no more results, storing all the results in a single list for us to look at when it's done. You could also do some evaluation and take some action in this loop if you were looking for something like if the users were set to pwnotrequired or pwneverexpires or something.
results=[]
results+=conn.response
while conn.result['controls']['1.2.840.113556.1.4.841']['value']['more_results'] == True:
  dirsync['Cookie']=conn.result['controls']['1.2.840.113556.1.4.841']['value']['cookie']
  dirsync_control = ldap3.protocol.controls.build_control('1.2.840.113556.1.4.841', True, dirsync)
  conn.search(
    default,
    '(objectclass=group)',
    paged_size = 1000,
    attributes=['member','name'],
    controls=(dirsync_control,)
  )
  results+=conn.response

Once we're all out of results to iterate through, the last cookie returned is the one we want to store for later, like so:
with open(cookie,'wb') as f:
  f.write(conn.result['controls']['1.2.840.113556.1.4.841']['value']['cookie'])

Code Chunks

Here are the two code chunks, one for powershell and one for python if you want them kinda all in one place:
$cookie="C:\dev\cookie"
$cred=get-credential

#The path to our ldap server
$ldap_path="LDAP://domaincontroller01.contoso.com:389/DC=contoso,DC=com"

#Our filter
$filter="(objectclass=group)"

#The properties we want - This is important since these are the properties we'll be watching for changes
$properties=@("member")

#Now we're into the meat - we create a .NET DirectoryEntry with our provided username and password
$entry=new-object System.DirectoryServices.DirectoryEntry($ldap_path, $cred.username, $cred.GetNetworkCredential().password)

#Then we declare a new directorysearcher from our entry and give it our filter and properties
$searcher=new-object System.DirectoryServices.DirectorySearcher($entry, $filter, $properties)

#Create a DIRSYNC control object
$sync_control=new-object System.DirectoryServices.DirectorySynchronization

#And assign the IncrementalValues flag to it. This makes it so that multivalue attributes (like member) will only show us changes instead of all the values
$sync_control.Option=[DirectoryServices.Protocols.DirectorySynchronizationOptions]::IncrementalValues

#Check if we have a cookie written, if we do - assign it
if(Test-Path $cookie) {$sync_control.ResetDirectorySynchronizationCookie(([System.IO.File]::ReadAllBytes($cookie)))}

#Grab our changes - which this time we don't really care that much about
$changes=$searcher.findAll()

#Get the cookie from our response
$thinmint = $sync_control.GetDirectorySynchronizationCookie()

#And store it away in our file path
[System.IO.File]::WriteAllBytes($cookie, $thinmint)


#Instantiate some credentials and a cookie path
import ldap3, getpass
cookiepath="C:\\dev\\cookie"
acctname="contoso\\MikeRyan-Admin"
password=getpass.getpass()

server = ldap3.Server(
  host="domaincontroller01.contoso.com",
  get_info = ldap3.ALL,
  port= 389
)

conn = ldap3.Connection(
  server=server,
  user=acctname,
  password=password,
  auto_bind = True
)

default=server.info.raw['defaultNamingContext'][0].decode()

#Import some structures from the python ASN1 implementation
from pyasn1.type.univ import Sequence, Integer, Boolean, OctetString
from pyasn1.type.namedtype import NamedTypes, NamedType
#Define a Sequence class with the different
from pyasn1.type.univ import Sequence, Integer, Boolean, OctetString
from pyasn1.type.namedtype import NamedTypes, NamedType
class dirsync_control_builder(Sequence):
  componentType = NamedTypes(
    NamedType('Flags', Integer()),
    NamedType('maxAttributeCount', Integer()),
    NamedType('Cookie', OctetString()),
  )

#Then we go in and define the properties of our new sequence object
dirsync = dirsync_control_builder()
dirsync['Flags'] = 0
dirsync['maxAttributeCount'] = 1000
try:
  with open(cookie,"rb") as f:
    dirsync['Cookie']=f.read()
except FileNotFoundError:
  dirsync['Cookie'] = ''

dirsync_control = ldap3.protocol.controls.build_control('1.2.840.113556.1.4.841', True, dirsync) conn.search(   default,
  '(objectclass=user)',
  paged_size = 1000,
  attributes=['useraccountcontrol'],
  controls=(dirsync_control,)
)

results=[]
results+=conn.response
while conn.result['controls']['1.2.840.113556.1.4.841']['value']['more_results'] == True:
  dirsync['Cookie']=conn.result['controls']['1.2.840.113556.1.4.841']['value']['cookie']
  dirsync_control = ldap3.protocol.controls.build_control('1.2.840.113556.1.4.841', True, dirsync)
  conn.search(
    default,
    '(objectclass=group)',
    paged_size = 1000,
    attributes=['member','name'],
    controls=(dirsync_control,)
  )
  results+=conn.response


with open(cookie,'wb') as f:
  f.write(conn.result['controls']['1.2.840.113556.1.4.841']['value']['cookie'])