Follow LDAP referrals for enumeration operations in multi-DC forests - #12
Open
j4s0nmo0n wants to merge 2 commits into
Open
Follow LDAP referrals for enumeration operations in multi-DC forests#12j4s0nmo0n wants to merge 2 commits into
j4s0nmo0n wants to merge 2 commits into
Conversation
ADWS returns Win32ErrorCode 8235 (ERROR_DS_REFERRAL) when a DC does not host the partition targeted by a request. Previously this surfaced as an opaque XML dump and killed the operation. This change makes the ADWS client swap to the referred DC and retry, transparently to callers. Design: * ADWSConnect._swap_endpoint(target_dc, hops) reconnects the NMF transport to another DC, updates self._fqdn (used in SOAP payloads), and is bounded by ADWSConnect.MAX_REFERRAL_HOPS (default 3) to prevent loops on misconfigured forests. * ADWSConnect.put() catches ADWSReferralError, swaps, rebuilds the payload with the new fqdn, and retries. This transparently covers --rbcd, --asrep, --spn, --disable-account, --dns-modify, --dns-remove, --dns-tombstone, --dns-resurrect, and --shadow-creds add/remove. * ad_dns_manager_adws._pull_with_referral_follow wraps pull_client.pull with the same retry pattern. Used by find_dns_node, add_dns_record's dnsZone enumeration, and shadow_credentials target discovery. * add_dns_record_adws, remove_dns_record_adws, and _set_dnstombstoned have their raw SOAP send/recv wrapped with a retry loop that rebuilds the payload after each swap. * add_dns_record_adws and add_computer proactively discover the DC that hosts the target container (via a lightweight pull) and route the subsequent Create there, avoiding CouldntFindParentObjectForCreation which AD returns instead of a referral when the parent DN doesn't exist on the queried DC. * getAccountDN can now return the fqdn of the DC that resolved the account, letting delete_computer route the Delete to the right DC. Scope: write operations only. Enumerations (--users, --computers, --groups) require restarting the whole Enumerate+Pull cycle after swap because the enumeration context is invalidated; that is out of scope here and will be addressed separately. Tested end-to-end in a parent/child forest (jjk.local / modulo.jjk.local): * --dns-add creates the record on the child DC via the parent DC * --shadow-creds list/add reach the target across the referral * --addcomputer creates the machine account in the child domain from the parent DC
This is the read-side complement to the write-side referral handling introduced in logangoins#11. It wraps the enumeration call site in soa.py so --users, --computers, --groups (and any other option that goes through the ADWSConnect.pull() enumeration path) transparently retry on the DC that hosts the target partition when the connected DC returns an LDAP referral. Approach The enumeration protocol makes an in-place swap impractical: the enumeration context (enum_ctx) issued by the Enumerate request is scoped to that DC and is invalidated by any Pull sent to another one (the referred DC returns NoSuchEnumCtxGuidExists). Reusing the _swap_endpoint mechanism from logangoins#11 mid-enumeration would break the pagination state. Instead, on ADWSReferralError we open a fresh ADWSConnect pull_client against the referred DC and restart the enumeration from scratch on that new client. Subsequent SID batches within the same query reuse the referred client (referred_client is set once, then all following iterations of the SID batch loop go through it) since re-attempting on the original DC would just produce the same referral. Scope * Applies only to the enumeration call site in soa.py (the SID batch loop at run_cli). This is the code path for --users, --computers, --groups, --asreproastable, --rbcds and the other options that use the paginated pull. * The smaller pulls in getAccountDN and friends were already covered by the pull_with_referral_follow helper introduced in logangoins#11. * No change to adws.py: the swap-then-restart approach is intentionally a caller-level pattern to preserve the internal invariants of ADWSConnect.pull() (checkpoints, SID resumption, renewal timer). Requirements The referred DC hostname (extracted from ADWSReferralError.target_dc) must resolve. When it does not, the retry surfaces a socket.gaierror with an actionable message pointing to /etc/hosts or -dc. Testing Lab: parent/child intra-forest (jjk.local + modulo.jjk.local), account in modulo, connection to dc-curse.jjk.local (parent DC). * --groups collected 63 objects from modulo across 6 pages * --computers collected 4 objects (child DC + 3 test machines) * --users on jjk.local as sanity check: no referral, unchanged behaviour Depends on logangoins#11 introduces ADWSReferralError and its target_dc attribute, both reused here.
Owner
|
This is great! An amazing QoL improvement for large scale enumeration. Give me a bit to review and test, and I'll get back to you. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is the read-side complement to the write-side referral handling in #11. It wraps the enumeration call site in
soa.pyso all enumeration options transparently retry on the DC that hosts the target partition when the connected DC returns an LDAP referral.Approach
The enumeration protocol makes an in-place swap impractical: the enumeration context (
enum_ctx) issued by the Enumerate request is scoped to that DC and is invalidated by any Pull sent to another one (the referred DC returnsNoSuchEnumCtxGuidExists). Reusing the_swap_endpointmechanism from #11 mid-enumeration would break the pagination state.Instead, on
ADWSReferralErrorwe open a freshADWSConnectpull_client against the referred DC and restart the enumeration from scratch on that new client. Subsequent SID batches within the same query reuse the referred client since re-attempting on the original DC would just produce the same referral.Scope
Applies to the enumeration call site in
soa.py(the SID batch loop atrun_cli), which is the code path for every option that goes through the paginated pull:--users,--computers,--groups--admins,--spns,--asreproastable,--rbcds--constrained,--unconstrained-q / --query(raw LDAP queries)The smaller pulls in
getAccountDNand related helpers were already covered by the_pull_with_referral_followhelper from #11.No change to
adws.py: the swap-then-restart approach is intentionally a caller-level pattern to preserve the internal invariants ofADWSConnect.pull()(checkpoints, SID resumption, renewal timer).Tested end-to-end
Lab: parent/child intra-forest (
jjk.local+modulo.jjk.local), account inmodulo, connection todc-curse.jjk.local(parent DC).--groups:
[!] LDAP referral: partition is hosted on modulo.jjk.local [*] Restarting enumeration on modulo.jjk.local (cross-domain: the enum_ctx is invalidated on swap, so a fresh Enumerate is required) [*] Connecting to 192.168.56.102 for resource:Enumeration [+] ADWS data collection complete: 63 objects saved to .soapy_data --computers:
[+] ADWS data collection complete: 4 objects saved to .soapy_data distinguishedName: CN=SHINJUKU,OU=Domain Controllers,DC=modulo,DC=jjk,DC=local distinguishedName: CN=TESTPC01,CN=Computers,DC=modulo,DC=jjk,DC=local ... Sanity check:
--usersagainstjjk.local(single-domain, connected DC hosts the target partition) shows no referral messages and preserves the original behaviour.Requirements
The referred DC hostname (extracted from
ADWSReferralError.target_dc) must resolve. When it does not, the retry surfaces asocket.gaierrorwith an actionable message pointing to/etc/hostsor-dc.