[cdac] .NET10: Implement DAC APIs used by CLRMA #108553
Assignees
Labels
area-Diagnostics-coreclr
enhancement
Product code improvement that does NOT require public API changes/additions
tracking
This issue is tracking the completion of other related issues.
Milestone
Comments
|
Tagging subscribers to this area: @tommcdon |
35 tasks
tommcdon
added
the
enhancement
lambdageek
added
the
tracking
7 tasks
lambdageek
added a commit
that referenced
this issue
Oct 17, 2024
#108685) RangeSectionMap is a data structure for coarse-grained lookups to man native code pointers to managed method descriptors that cover the entire addressable memory. This PR adds unit tests for the cdac reader RangeSectionMap lookup algorithm and some unit tests for the ExecutionManager contract APIs that use a RangeSectionMap and NibbleMap together to do a full TargetCodePointer->MethodDesc lookup Contributes to #108553 * [cdac] ExecutionManager contract and lookup map tests * add RangeSectionMap docs * exhaustively test RangeSectionMap.GetIndexForLevel * fix lookup in RangeSection.Find RangeSectionLookupAlgorithm.FindFragment finds the slot containing the range section fragment pointer. Have to dereference it once to get the actual RangeSectionFragment pointer from the slot. This "worked" before because RangeSectionFragment.Next is at offset 0, so the first lookup would have a garbage range, so we would follow the "next" field and get to the correct fragment * make a testable RangeSectionMap.FindFragmentInternal * brief nibble map summary * [cdac] Implement NibbleMap lookup and tests The execution manager uses a nibble map to quickly map program counter pointers to the beginnings of the native code for the managed method. Implement the lookup algorithm for a nibble map. Start adding unit tests for the nibble map Also for testing in MockMemorySpace simplify ReaderContext, there's nothing special about the descriptor HeapFragments anymore. We can use a uniform reader. * NibbleMap: fix bug and add test Previously we incorrectly computed the prior map index when doing the backward linear search * [testing] display Target values in hex in debugger views * MockMemorySpace: simplify ReaderContext there's nothing special about the descriptor HeapFragments anymore. We can use a uniform reader * refactor ExecutionManager * ExecutionManager contract the presence of RangeSection.R2RModule is a discriminator for whether we're looking at EE code or R2R code * use better cdac_data friends * markdown cleanup * document the ExecutionManager methods * cache CodeBlock based on given code pointer, not start of the method The CodeBlock includes the relative offset (given ip - start of method) so it's not ok to share for different code pointers into the same method * Make TestPlaceholderTarget data cache more useful; make TestRegistry lazy * bugfix: StubCodeBlockLast is uint8 * add a simple bump allocator to MockMemorySpace * Add a field layout algorithm to TargetTestHelpers * move all the test builders to a separate class * cleanup test infra * remove a few more magic constants * EECodeInfo -> CodeBlock * EffectiveBitsForLevel -> GetIndexForLevel * add documentation to managed contract impl * RSLATestTarget -> RSMTestTarget --------- Co-authored-by: Elinor Fung <[email protected]>
lambdageek
added a commit
to lambdageek/runtime
that referenced
this issue
Oct 17, 2024
The CodeVersions contract implements the IL and native code versioning as described in [code-versioning.md](docs/design/features/code-versioning.md) Contributes to dotnet#108553 Contributes to dotnet#99302 * rename contract NativeCodePointers => CodeVersions * FindActiveILCodeVersion * implement GetModuleLookupMapElement * FindActiveILCodeVersion/FindActiveNativeCodeVersion * il code version lookup table * remove AppDomain.CodeVersionManager from cdac * CodeVersionManager is basically a static class in the C++ side * NativeCodeVersionContract.GetSpecificNativeCodeVersion * checkpoint: start adding NativeCodeVersion operations * WIP: native code version
lambdageek
added a commit
that referenced
this issue
Oct 18, 2024
* [cdac] Add a CodeVersions contract The CodeVersions contract implements the IL and native code versioning as described in [code-versioning.md](docs/design/features/code-versioning.md) Contributes to #108553 Contributes to #99302 * rename contract NativeCodePointers => CodeVersions * FindActiveILCodeVersion * implement GetModuleLookupMapElement * FindActiveILCodeVersion/FindActiveNativeCodeVersion * il code version lookup table * remove AppDomain.CodeVersionManager from cdac * CodeVersionManager is basically a static class in the C++ side * NativeCodeVersionContract.GetSpecificNativeCodeVersion * checkpoint: start adding NativeCodeVersion operations * WIP: native code version * tighter cdac_data friends * remove unused DataType values they'll come in from a future PR * move ecma metadata helpers to their own class * WIP: CodeVersionsTests * WIP: TestGetNativeCodeVersionOneVersionVersionable * implement GetNativeCode for a NativeCodeVersionNode handle * checkpoint: TestGetNativeCodeVersionOneVersionVersionable passes * Add TestGetActiveNativeCodeVersionDefaultCase * don't add ILCodeVersioningState::Next field yet * describe Loader::GetModuleLookupMapElement contract * remove unused member * update contract markdown * simplify flags handling a bit * add TODOs for RuntimeTypeSystem additions
lambdageek
added a commit
that referenced
this issue
Oct 24, 2024
Adds a `PrecodeStubs` contract for working with precode. Adds a `CDacMetadata` contract for exposing useful platform-specific properties Contributes to #99302 Contributes to #108553 * [cdac] PrecodeStubs contract * move PrecodeStubs to a separate contract * remove instance field initialization from PrecodeMachineDescriptor * PrecodeMachineDescriptor::Init() will initialize g_PrecodeMachineDescriptor * fix thumb flag logic * add FixupPrecodeData * checkpoint: StubPrecodeData, StubCodePageSize * checkpoint: PrecodeMachineDescriptor and KnownPrecodeType * WIP: start adding PrecodeStubs test infrastructure * checkpoint: stub precode test passing * set the "thumb" bit * cleanup thumb bit * add a CDacMetadata struct * permute PrecodeMachineDescriptor fields for better packability * reader for CDacMetadata contract * WIP: code pointer flags * checkpoint tests with CDacMetadata * rename CDacMetadata -> PlatformMetadata * add a define for Loongarch64 stub precode shift * conditionally define fields for precode absent precode types * clarify how stub and pinvoke precodes work --------- Co-authored-by: Elinor Fung <[email protected]>
This was referenced Oct 25, 2024
lambdageek
added a commit
that referenced
this issue
Nov 1, 2024
This is enough for `!PrintException` without R2R methods on the stack There's also a `ReJIT` contract here which just checks whether rejit is enabled. Most of the complexity is in validating MethodDescs Contributes to #99302 Contributes to #108553 --- * ReJIT contract * document the ExecutionManager methods * cache EECodeInfo based on given code pointer, not start of the method The EECodeInfo includes the relative offset (given ip - start of method) so it's not ok to share for different code pointers into the same method * add legacy DAC comparison for GetMethodDescData * add documentation to managed contract impl * add TODOs for RuntimeTypeSystem additions * update contract markdown * get collectible flag for module from Assembly instead of LoaderAllocator * Use CodePointerFlags in RuntimeTypeSystem * implement MethodDesc GetLoaderModule via the loader module attached to a chunk * implement MethodTable GetLoaderModule * add MethodTable.AuxiliaryData field to test typeinfo * implement vtable indirections * fixup: GetVtableIndirections - add MethodTable size to data descriptor * rename some vestigial CDacMetadata references * WIP: MethodDesc tests * checkpoint GetMethodToken test passes * move MethodValidation to a separate class also move method flags out of the RuntimeTypeSystem_1 contract for the cases where the method validation needs to call back to type validation, go via the contract * move COR_PRF_MONITOR into ReJIT_1 it's an implementation detail * Update src/native/managed/cdacreader/src/Legacy/SOSDacImpl.cs Co-authored-by: Aaron Robinson <[email protected]> * assert the expected slot number * Exercise a few more easy properties of MethodDesc * remove TypeHandleFromAddress from contract we just use GetTypeHandle now * clean up MethodClassification -> size lookups * document RuntimeTypeSystem contract additions * update the CodeVersions contract to match implementation * remove some TODO comments * add sizes for method desc subclasses the "additional pointers" logic in RuntimeTypeSystem_1 depends on the sizes --------- Co-authored-by: Elinor Fung <[email protected]> Co-authored-by: Aaron Robinson <[email protected]>
mikelle-rogers
pushed a commit
to mikelle-rogers/runtime
that referenced
this issue
Dec 10, 2024
dotnet#108685) RangeSectionMap is a data structure for coarse-grained lookups to man native code pointers to managed method descriptors that cover the entire addressable memory. This PR adds unit tests for the cdac reader RangeSectionMap lookup algorithm and some unit tests for the ExecutionManager contract APIs that use a RangeSectionMap and NibbleMap together to do a full TargetCodePointer->MethodDesc lookup Contributes to dotnet#108553 * [cdac] ExecutionManager contract and lookup map tests * add RangeSectionMap docs * exhaustively test RangeSectionMap.GetIndexForLevel * fix lookup in RangeSection.Find RangeSectionLookupAlgorithm.FindFragment finds the slot containing the range section fragment pointer. Have to dereference it once to get the actual RangeSectionFragment pointer from the slot. This "worked" before because RangeSectionFragment.Next is at offset 0, so the first lookup would have a garbage range, so we would follow the "next" field and get to the correct fragment * make a testable RangeSectionMap.FindFragmentInternal * brief nibble map summary * [cdac] Implement NibbleMap lookup and tests The execution manager uses a nibble map to quickly map program counter pointers to the beginnings of the native code for the managed method. Implement the lookup algorithm for a nibble map. Start adding unit tests for the nibble map Also for testing in MockMemorySpace simplify ReaderContext, there's nothing special about the descriptor HeapFragments anymore. We can use a uniform reader. * NibbleMap: fix bug and add test Previously we incorrectly computed the prior map index when doing the backward linear search * [testing] display Target values in hex in debugger views * MockMemorySpace: simplify ReaderContext there's nothing special about the descriptor HeapFragments anymore. We can use a uniform reader * refactor ExecutionManager * ExecutionManager contract the presence of RangeSection.R2RModule is a discriminator for whether we're looking at EE code or R2R code * use better cdac_data friends * markdown cleanup * document the ExecutionManager methods * cache CodeBlock based on given code pointer, not start of the method The CodeBlock includes the relative offset (given ip - start of method) so it's not ok to share for different code pointers into the same method * Make TestPlaceholderTarget data cache more useful; make TestRegistry lazy * bugfix: StubCodeBlockLast is uint8 * add a simple bump allocator to MockMemorySpace * Add a field layout algorithm to TargetTestHelpers * move all the test builders to a separate class * cleanup test infra * remove a few more magic constants * EECodeInfo -> CodeBlock * EffectiveBitsForLevel -> GetIndexForLevel * add documentation to managed contract impl * RSLATestTarget -> RSMTestTarget --------- Co-authored-by: Elinor Fung <[email protected]>
mikelle-rogers
pushed a commit
to mikelle-rogers/runtime
that referenced
this issue
Dec 10, 2024
* [cdac] Add a CodeVersions contract The CodeVersions contract implements the IL and native code versioning as described in [code-versioning.md](docs/design/features/code-versioning.md) Contributes to dotnet#108553 Contributes to dotnet#99302 * rename contract NativeCodePointers => CodeVersions * FindActiveILCodeVersion * implement GetModuleLookupMapElement * FindActiveILCodeVersion/FindActiveNativeCodeVersion * il code version lookup table * remove AppDomain.CodeVersionManager from cdac * CodeVersionManager is basically a static class in the C++ side * NativeCodeVersionContract.GetSpecificNativeCodeVersion * checkpoint: start adding NativeCodeVersion operations * WIP: native code version * tighter cdac_data friends * remove unused DataType values they'll come in from a future PR * move ecma metadata helpers to their own class * WIP: CodeVersionsTests * WIP: TestGetNativeCodeVersionOneVersionVersionable * implement GetNativeCode for a NativeCodeVersionNode handle * checkpoint: TestGetNativeCodeVersionOneVersionVersionable passes * Add TestGetActiveNativeCodeVersionDefaultCase * don't add ILCodeVersioningState::Next field yet * describe Loader::GetModuleLookupMapElement contract * remove unused member * update contract markdown * simplify flags handling a bit * add TODOs for RuntimeTypeSystem additions
mikelle-rogers
pushed a commit
to mikelle-rogers/runtime
that referenced
this issue
Dec 10, 2024
Adds a `PrecodeStubs` contract for working with precode. Adds a `CDacMetadata` contract for exposing useful platform-specific properties Contributes to dotnet#99302 Contributes to dotnet#108553 * [cdac] PrecodeStubs contract * move PrecodeStubs to a separate contract * remove instance field initialization from PrecodeMachineDescriptor * PrecodeMachineDescriptor::Init() will initialize g_PrecodeMachineDescriptor * fix thumb flag logic * add FixupPrecodeData * checkpoint: StubPrecodeData, StubCodePageSize * checkpoint: PrecodeMachineDescriptor and KnownPrecodeType * WIP: start adding PrecodeStubs test infrastructure * checkpoint: stub precode test passing * set the "thumb" bit * cleanup thumb bit * add a CDacMetadata struct * permute PrecodeMachineDescriptor fields for better packability * reader for CDacMetadata contract * WIP: code pointer flags * checkpoint tests with CDacMetadata * rename CDacMetadata -> PlatformMetadata * add a define for Loongarch64 stub precode shift * conditionally define fields for precode absent precode types * clarify how stub and pinvoke precodes work --------- Co-authored-by: Elinor Fung <[email protected]>
mikelle-rogers
pushed a commit
to mikelle-rogers/runtime
that referenced
this issue
Dec 10, 2024
This is enough for `!PrintException` without R2R methods on the stack There's also a `ReJIT` contract here which just checks whether rejit is enabled. Most of the complexity is in validating MethodDescs Contributes to dotnet#99302 Contributes to dotnet#108553 --- * ReJIT contract * document the ExecutionManager methods * cache EECodeInfo based on given code pointer, not start of the method The EECodeInfo includes the relative offset (given ip - start of method) so it's not ok to share for different code pointers into the same method * add legacy DAC comparison for GetMethodDescData * add documentation to managed contract impl * add TODOs for RuntimeTypeSystem additions * update contract markdown * get collectible flag for module from Assembly instead of LoaderAllocator * Use CodePointerFlags in RuntimeTypeSystem * implement MethodDesc GetLoaderModule via the loader module attached to a chunk * implement MethodTable GetLoaderModule * add MethodTable.AuxiliaryData field to test typeinfo * implement vtable indirections * fixup: GetVtableIndirections - add MethodTable size to data descriptor * rename some vestigial CDacMetadata references * WIP: MethodDesc tests * checkpoint GetMethodToken test passes * move MethodValidation to a separate class also move method flags out of the RuntimeTypeSystem_1 contract for the cases where the method validation needs to call back to type validation, go via the contract * move COR_PRF_MONITOR into ReJIT_1 it's an implementation detail * Update src/native/managed/cdacreader/src/Legacy/SOSDacImpl.cs Co-authored-by: Aaron Robinson <[email protected]> * assert the expected slot number * Exercise a few more easy properties of MethodDesc * remove TypeHandleFromAddress from contract we just use GetTypeHandle now * clean up MethodClassification -> size lookups * document RuntimeTypeSystem contract additions * update the CodeVersions contract to match implementation * remove some TODO comments * add sizes for method desc subclasses the "additional pointers" logic in RuntimeTypeSystem_1 depends on the sizes --------- Co-authored-by: Elinor Fung <[email protected]> Co-authored-by: Aaron Robinson <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
See DAC interfaces used by CLRMA in the diagnostics repo.
This is a follow-up to #99302, contributes to #99298
!PrintExceptionsupportGetMethodDescDataIXCLRDataModuleimplementation: Module name fallback if debugger and GetPEFileName() can't get the name.ISOSDacInterface::GetModuleandIXCLRDataModule::GetFileName#109133ISOSDacInterface::GetModuleandIXCLRDataModule::GetFileName#109133IXCLRDataProcess::GetTaskByOSThreadID#109230IXCLRDataTask::CreateStackWalk#109350A related issue is #108720 tracking the work to deploy cdac with SOS
The text was updated successfully, but these errors were encountered: