Archived from groups: microsoft.public.active.directory.interfaces,microsoft.public.adsi.general,microsoft.public.win2000.active_directory (More info?)

I'm interested in identifying the programming interfaces used for real-time
auditing of changes to objects in Active Directory. The LDAP uSNChanged and
DirSync ADSI control are not sufficient for what I'm doing, and modifying
the object security and DC security policies to turn on security auditing is
not a viable alternative, either.

There are existing products, such as Quest's "Quest Change Manager for
Active Directory" that appear to be collecting the same time of real-time
information that I'm looking for, and Quest claims to be doing so w/o using
the native audit log features of Windows when doing so.

In a nutshell, I need to receive notifications on a DC whenever an event of
interest happens within the domain or any of its child containers that the
DC contains in its replica of its portion of the tree. I would prefer to
register to receive notification of only the events I'm interested in, but
if I have to receive all events and evaluate them that's OK, too.

Object creation
Object deletion
Object modification [excluding DN changes]
Object rename/move

In the case of object modification, I need to know what attribute was
changed, what the previous value was and what the new value is, or, if it is
a multi-valued attribute, I need to know what the particular value is that
was added to or removed from the list along with the actual add/remove value
operation being identified.

I've gone over the Platform SDK docs [updated for Windows Server 2003 SP1]
and I'm not seeing *anything* even remotely close to what I'm looking for.
However, since there are commercial products on the market that seem to be
obtaining the same type of information, there's got to be some sort of
programming interface with which to obtain the desired information. In the
Novell environment eDirectory [f.k.a. NDS] has a very comprehensive event
monitoring API that can be used to achieve a fine degree of granularity in
terms of the events that can be monitored, and the event notifications can
be delivered via an async callback mechanism.

Is there something obvious or less than obvious that I'm missing? Or, are
these products making use of undocumented interfaces to perform their tasks?

Any assistance would be appreciated.


TIA,

Chuck
--
Chuck Chopp

ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com

RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
103 Autumn Hill Road 864 801 2774 fax
Greer, SC 29651

Do not send me unsolicited commercial email.

Archived from groups: microsoft.public.active.directory.interfaces,microsoft.public.adsi.general,microsoft.public.win2000.active_directory (More info?)

If you don't want to do change polling, then the only option I'm aware of
are LDAP change notifications. They are documented here:

http://msdn.microsoft.com/library/ [...] frame=true

Note that MS warns against using these on an entire naming context due to
performance problems, especially on big DCs. It appears that they really
want you to use one of the polling-based approaches (dirsync or usnChanged),
even though you have ruled that out.

If anyone else has heard of anything, I'd be interested in hearing about it.

Joe K.

"Chuck Chopp" <ChuckChopp@rtfmcsi.com> wrote in message
news:e%23vsW00gFHA.2700@TK2MSFTNGP15.phx.gbl...
> I'm interested in identifying the programming interfaces used for
> real-time auditing of changes to objects in Active Directory. The LDAP
> uSNChanged and DirSync ADSI control are not sufficient for what I'm doing,
> and modifying the object security and DC security policies to turn on
> security auditing is not a viable alternative, either.
>
> There are existing products, such as Quest's "Quest Change Manager for
> Active Directory" that appear to be collecting the same time of real-time
> information that I'm looking for, and Quest claims to be doing so w/o
> using the native audit log features of Windows when doing so.
>
> In a nutshell, I need to receive notifications on a DC whenever an event
> of interest happens within the domain or any of its child containers that
> the DC contains in its replica of its portion of the tree. I would prefer
> to register to receive notification of only the events I'm interested in,
> but if I have to receive all events and evaluate them that's OK, too.
>
> Object creation
> Object deletion
> Object modification [excluding DN changes]
> Object rename/move
>
> In the case of object modification, I need to know what attribute was
> changed, what the previous value was and what the new value is, or, if it
> is a multi-valued attribute, I need to know what the particular value is
> that was added to or removed from the list along with the actual
> add/remove value operation being identified.
>
> I've gone over the Platform SDK docs [updated for Windows Server 2003 SP1]
> and I'm not seeing *anything* even remotely close to what I'm looking for.
> However, since there are commercial products on the market that seem to be
> obtaining the same type of information, there's got to be some sort of
> programming interface with which to obtain the desired information. In
> the Novell environment eDirectory [f.k.a. NDS] has a very comprehensive
> event monitoring API that can be used to achieve a fine degree of
> granularity in terms of the events that can be monitored, and the event
> notifications can be delivered via an async callback mechanism.
>
> Is there something obvious or less than obvious that I'm missing? Or, are
> these products making use of undocumented interfaces to perform their
> tasks?
>
> Any assistance would be appreciated.
>
>
> TIA,
>
> Chuck
> --
> Chuck Chopp
>
> ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com
>
> RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
> 103 Autumn Hill Road 864 801 2774 fax
> Greer, SC 29651
>
> Do not send me unsolicited commercial email.

Archived from groups: microsoft.public.active.directory.interfaces,microsoft.public.adsi.general,microsoft.public.win2000.active_directory (More info?)

Joe Kaplan (MVP - ADSI) wrote:

> If you don't want to do change polling, then the only option I'm aware of
> are LDAP change notifications. They are documented here:
>
> http://msdn.microsoft.com/library/ [...] frame=true

Yes, that's the stuff straight out of the MSDN Platform SDK... and it's very
much lacking in terms of what I'm wanting to do. Perhaps I'm spoiled with
what can be done w/respect to event notification in eDirectory, but somehow
someway AD has got to have a "native interface" that exposes better
functionality than what's available via LDAP. Even the LDAP control
extensions that AD implements are weak in comparison to the ones implemented
by eDirectory.

> Note that MS warns against using these on an entire naming context due to
> performance problems, especially on big DCs. It appears that they really
> want you to use one of the polling-based approaches (dirsync or usnChanged),
> even though you have ruled that out.

I've come to the conclusion that Microsoft has very little faith in its own
directory services product compared to what Novell does with eDirectory.
It's kind of like the difference between a Fisher Price toddler's piano and
a Steinway baby grand piano... one is a toy for children and the other is a
finely tuned professional instrument. That's not said to start a flame-war,
it's simply an observation and vented in frustration at the lack of
documented & supported functionality. For small tasks, AD works just fine,
but for large scale industrial-strength directory-enabled applications, MS
seems to be hesitant in terms of what AD will be capable of doing.

> If anyone else has heard of anything, I'd be interested in hearing about it.


The LDAP method mentioned in the Platform SDK doesn't provide the
granularity I'm looking for, nor do either of the polling methods.
Specifically, I need to know if the change is due to object creation,
deletion, rename, move or is just a generic modification of the object's
attributes. And, if it's modified attributes, I need to know the before &
after attribute values for single valued attributes, and, for multi-valued
attributes, I need to know the individual value in the list that was
modified and whether the value was added to or removed from the list.

Take a look at these links:

http://wm.quest.com/Library/getDoc [...] et=cmadpds
http://www.bi101.net/products/solutions/netpro/
http://www.netpro.com/products/changeauditor/index.cfm

These products are all making claims of auditing AD events and offering a
fine level of granularity in the changes w/o making use of any of the
built-in auditing mechanisms. The functionality they describe cannot be
achieved using DirSync or LDAP as far as I know, so that leaves me with the
thought that they are using some *other* interface into Active Directory.
It's that *other* interface that I'm interested in learning about. Given
that there's more than one product doing this, I'm guessing that they all
work in a similar manner using the same interface into AD. The alternative
is that they're maintaining private replicas of AD information, and that's a
grossly inefficient method that wouldn't perform nearly as well how these
products are supposed to be performing.

Maybe there's a means of hooking into the replication interfaces in AD. If
I could reliably hook into AD in that manner then I could intercept every
single piece of replication traffic and *that* would allow me to obtain the
desired information in real-time, or at least as close to real-time as the
replication schedule allows for. It would eliminate polling and it would
certainly allow me to directly observe in very fine detail exactly what is
going on in AD.

Another possibility is the thinly documented event tracing facility.
There's a very vague reference to in in the Platform SDK in connection with
AD, but there's nothing of substance in the docs to indicate if I'm heading
in the right direction with the thoughts of trying to track down an event
source that will provide the desired event information.


--
Chuck Chopp

ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com

RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
103 Autumn Hill Road 864 801 2774 fax
Greer, SC 29651

Do not send me unsolicited commercial email.

Archived from groups: microsoft.public.active.directory.interfaces,microsoft.public.adsi.general,microsoft.public.win2000.active_directory (More info?)

Like I said, I can't tell you any more nor do I have any idea how those
other products are doing this. I think you need someone from MS to weigh
in.

You might want to get one of those registered no-spam aliases and try
posting again to see f you can coax an answer out of them.

http://msdn.microsoft.com/newsgroups/managed/

Joe K.

"Chuck Chopp" <ChuckChopp@rtfmcsi.com> wrote in message
news:eWKZxT3gFHA.2560@TK2MSFTNGP10.phx.gbl...
> Joe Kaplan (MVP - ADSI) wrote:
>
>> If you don't want to do change polling, then the only option I'm aware of
>> are LDAP change notifications. They are documented here:
>>
>> http://msdn.microsoft.com/library/ [...] frame=true
>
> Yes, that's the stuff straight out of the MSDN Platform SDK... and it's
> very much lacking in terms of what I'm wanting to do. Perhaps I'm spoiled
> with what can be done w/respect to event notification in eDirectory, but
> somehow someway AD has got to have a "native interface" that exposes
> better functionality than what's available via LDAP. Even the LDAP
> control extensions that AD implements are weak in comparison to the ones
> implemented by eDirectory.
>
>> Note that MS warns against using these on an entire naming context due to
>> performance problems, especially on big DCs. It appears that they really
>> want you to use one of the polling-based approaches (dirsync or
>> usnChanged), even though you have ruled that out.
>
> I've come to the conclusion that Microsoft has very little faith in its
> own directory services product compared to what Novell does with
> eDirectory. It's kind of like the difference between a Fisher Price
> toddler's piano and a Steinway baby grand piano... one is a toy for
> children and the other is a finely tuned professional instrument. That's
> not said to start a flame-war, it's simply an observation and vented in
> frustration at the lack of documented & supported functionality. For
> small tasks, AD works just fine, but for large scale industrial-strength
> directory-enabled applications, MS seems to be hesitant in terms of what
> AD will be capable of doing.
>
>> If anyone else has heard of anything, I'd be interested in hearing about
>> it.
>
>
> The LDAP method mentioned in the Platform SDK doesn't provide the
> granularity I'm looking for, nor do either of the polling methods.
> Specifically, I need to know if the change is due to object creation,
> deletion, rename, move or is just a generic modification of the object's
> attributes. And, if it's modified attributes, I need to know the before &
> after attribute values for single valued attributes, and, for multi-valued
> attributes, I need to know the individual value in the list that was
> modified and whether the value was added to or removed from the list.
>
> Take a look at these links:
>
> http://wm.quest.com/Library/getDoc [...] et=cmadpds
> http://www.bi101.net/products/solutions/netpro/
> http://www.netpro.com/products/changeauditor/index.cfm
>
> These products are all making claims of auditing AD events and offering a
> fine level of granularity in the changes w/o making use of any of the
> built-in auditing mechanisms. The functionality they describe cannot be
> achieved using DirSync or LDAP as far as I know, so that leaves me with
> the thought that they are using some *other* interface into Active
> Directory. It's that *other* interface that I'm interested in learning
> about. Given that there's more than one product doing this, I'm guessing
> that they all work in a similar manner using the same interface into AD.
> The alternative is that they're maintaining private replicas of AD
> information, and that's a grossly inefficient method that wouldn't perform
> nearly as well how these products are supposed to be performing.
>
> Maybe there's a means of hooking into the replication interfaces in AD.
> If I could reliably hook into AD in that manner then I could intercept
> every single piece of replication traffic and *that* would allow me to
> obtain the desired information in real-time, or at least as close to
> real-time as the replication schedule allows for. It would eliminate
> polling and it would certainly allow me to directly observe in very fine
> detail exactly what is going on in AD.
>
> Another possibility is the thinly documented event tracing facility.
> There's a very vague reference to in in the Platform SDK in connection
> with AD, but there's nothing of substance in the docs to indicate if I'm
> heading in the right direction with the thoughts of trying to track down
> an event source that will provide the desired event information.
>
>
> --
> Chuck Chopp
>
> ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com
>
> RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
> 103 Autumn Hill Road 864 801 2774 fax
> Greer, SC 29651
>
> Do not send me unsolicited commercial email.

Archived from groups: microsoft.public.active.directory.interfaces,microsoft.public.adsi.general,microsoft.public.win2000.active_directory (More info?)

Joe Kaplan (MVP - ADSI) wrote:

> Like I said, I can't tell you any more nor do I have any idea how those
> other products are doing this. I think you need someone from MS to weigh
> in.
>
> You might want to get one of those registered no-spam aliases and try
> posting again to see f you can coax an answer out of them.
>
> http://msdn.microsoft.com/newsgroups/managed/

It'd take opening a support incident, I think, as the depth of knowledge I'm
looking for is turning out not be likely to be found out on Usenet or the
web. What I've learned about the commercial auditing & change reporting
products that I referenced is that they are using unsupported methods to
directly tap into AD through the use of hooks that allow internal AD
functions to be intercepted. They do not make use of any of the documented
& supported methods for obtaining AD change notification.


--
Chuck Chopp

ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com

RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
103 Autumn Hill Road 864 801 2774 fax
Greer, SC 29651

Do not send me unsolicited commercial email.

Archived from groups: microsoft.public.active.directory.interfaces,microsoft.public.adsi.general,microsoft.public.win2000.active_directory (More info?)

Actually the dirsync control is a replication based control. It will not show
you all changes, only changes that would replicate. With it you sip from the
firehose and sort out what you need from it and realize that local
non-replicating changes will not be available through it.

The LDAP event notification is really not meant for monitoring all of AD but for
watching changes on specific pieces of AD just like the registry event
notification is really for watching specific pieces of the registry versus the
entire thing.

Last time I talked to the NetPro guys they were doing at least some of the work
with Event Tracing so your best bet would be to dig into that more. It isn't a
popular subject so you aren't likely to find much info. Those who have done it
are those who are selling products and will obviously be a bit slow to provide
source code or details. It wouldn't make sense if they spent money figuring it
out and then just handed it over to anyone asking.

I can't really speak to the what eDir can do versus what AD can do. It really
isn't relavent, we are talking about AD, not eDir. All of the complaints about
what one has over the other isn't going to change either nor make anything work.
If you need specific functionality out of AD, the mechanism is to submit a DCR
to Microsoft for the change through PSS. Expect that if there isn't a good
number of similar requests, it will most likely be dropped.

joe


--
Joe Richards Microsoft MVP Windows Server Directory Services
www.joeware.net


Chuck Chopp wrote:
> Joe Kaplan (MVP - ADSI) wrote:
>
>> If you don't want to do change polling, then the only option I'm aware
>> of are LDAP change notifications. They are documented here:
>>
>> http://msdn.microsoft.com/library/ [...] frame=true
>>
>
>
> Yes, that's the stuff straight out of the MSDN Platform SDK... and it's
> very much lacking in terms of what I'm wanting to do. Perhaps I'm
> spoiled with what can be done w/respect to event notification in
> eDirectory, but somehow someway AD has got to have a "native interface"
> that exposes better functionality than what's available via LDAP. Even
> the LDAP control extensions that AD implements are weak in comparison to
> the ones implemented by eDirectory.
>
>> Note that MS warns against using these on an entire naming context due
>> to performance problems, especially on big DCs. It appears that they
>> really want you to use one of the polling-based approaches (dirsync or
>> usnChanged), even though you have ruled that out.
>
>
> I've come to the conclusion that Microsoft has very little faith in its
> own directory services product compared to what Novell does with
> eDirectory. It's kind of like the difference between a Fisher Price
> toddler's piano and a Steinway baby grand piano... one is a toy for
> children and the other is a finely tuned professional instrument.
> That's not said to start a flame-war, it's simply an observation and
> vented in frustration at the lack of documented & supported
> functionality. For small tasks, AD works just fine, but for large scale
> industrial-strength directory-enabled applications, MS seems to be
> hesitant in terms of what AD will be capable of doing.
>
>> If anyone else has heard of anything, I'd be interested in hearing
>> about it.
>
>
>
> The LDAP method mentioned in the Platform SDK doesn't provide the
> granularity I'm looking for, nor do either of the polling methods.
> Specifically, I need to know if the change is due to object creation,
> deletion, rename, move or is just a generic modification of the object's
> attributes. And, if it's modified attributes, I need to know the before
> & after attribute values for single valued attributes, and, for
> multi-valued attributes, I need to know the individual value in the list
> that was modified and whether the value was added to or removed from the
> list.
>
> Take a look at these links:
>
> http://wm.quest.com/Library/getDoc [...] et=cmadpds
> http://www.bi101.net/products/solutions/netpro/
> http://www.netpro.com/products/changeauditor/index.cfm
>
> These products are all making claims of auditing AD events and offering
> a fine level of granularity in the changes w/o making use of any of the
> built-in auditing mechanisms. The functionality they describe cannot be
> achieved using DirSync or LDAP as far as I know, so that leaves me with
> the thought that they are using some *other* interface into Active
> Directory. It's that *other* interface that I'm interested in learning
> about. Given that there's more than one product doing this, I'm
> guessing that they all work in a similar manner using the same interface
> into AD. The alternative is that they're maintaining private replicas
> of AD information, and that's a grossly inefficient method that wouldn't
> perform nearly as well how these products are supposed to be performing.
>
> Maybe there's a means of hooking into the replication interfaces in AD.
> If I could reliably hook into AD in that manner then I could intercept
> every single piece of replication traffic and *that* would allow me to
> obtain the desired information in real-time, or at least as close to
> real-time as the replication schedule allows for. It would eliminate
> polling and it would certainly allow me to directly observe in very fine
> detail exactly what is going on in AD.
>
> Another possibility is the thinly documented event tracing facility.
> There's a very vague reference to in in the Platform SDK in connection
> with AD, but there's nothing of substance in the docs to indicate if I'm
> heading in the right direction with the thoughts of trying to track down
> an event source that will provide the desired event information.
>
>

Archived from groups: microsoft.public.active.directory.interfaces,microsoft.public.adsi.general,microsoft.public.win2000.active_directory (More info?)

Joe Richards [MVP] wrote:

> Last time I talked to the NetPro guys they were doing at least some of
> the work with Event Tracing so your best bet would be to dig into that
> more. It isn't a popular subject so you aren't likely to find much info.
> Those who have done it are those who are selling products and will
> obviously be a bit slow to provide source code or details. It wouldn't
> make sense if they spent money figuring it out and then just handed it
> over to anyone asking.

LOL - I know - At this point I'm simply trying to gather information for
purposes of doing further research into the subject. I've been able to rule
out a significant number of possible of avenues of research, so the scope is
being narrowed down to something manageable.

> I can't really speak to the what eDir can do versus what AD can do. It
> really isn't relavent, we are talking about AD, not eDir. All of the

The relevance is relative, perhaps? In this case... I'm dealing with a port
of code that was originally written in the Novell NDS/eDir environment and
I'm trying to find equivalent functionality in AD in terms of event
monitoring. I'd say the differences between the two directory services are
very relevant in terms of the feasibility of making the port successful.
And so I keep on with the research....


--
Chuck Chopp

ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com

RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
103 Autumn Hill Road 864 801 2774 fax
Greer, SC 29651

Do not send me unsolicited commercial email.