Feature #65: track metrics of failed tests #82
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
disabled (--no-monitor cmd flag) changelog updated in this commit too The issue was an exception not being raised when calling wrapped_function when monitoring is disabled in line 216. The raise is being handled now in the following lines 218ff.
to own internal module
use needed parameters
function to always return memory usage no matter if function throws exception or not
column TEST_PASSED (type boolean) to TEST_METRICS table to indicate if the test being logged passed.
column TEST_PASSED (type boolean) to TEST_METRICS table to indicate if the test being logged passed. Testing: Add proper test to test successful database migration
…rt/pytest-monitor into feature/track-metrics-of-failed
…rt/pytest-monitor into feature/track-metrics-of-failed Seems like an empty merge with no changes, had to be done to be able to push new commits upstream
…rt/pytest-monitor into feature/track-metrics-of-failed
…onfig on Initialization of DBHandler (test if new column existent)
file test_monitor_handler.py
specify the sql backend.
for Bitbucket CI. (Part of session or execution context table)
Updates the sqlite test functions to the newest versions that are also used in the PostgresDBHandler branch.
Merge newest main branch version into this branch
Add hint in profiler.py to explain why only the recently created MemTimer is being killed on failed tests throwing an exception.
The documentation files are updated to include the newest changes.
lhpt2
changed the title
added 4 commits
July 3, 2024 15:49
Returning exceptions inside the wrapped_function() can lead to assertions being ignored, therefore they need to be raised instantly and parent calls need to wrap their call in a try except block.
Merge refactored bugfix 79 into track-metrics-of-failed because it is dependent on this bugfix.
In order to handle all exceptions of a failing test, catch exceptions with BaseException in profiler.py.
lhpt2
pushed a commit
to einhundert/pytest-monitor
that referenced
this pull request
Jul 15, 2024
The memory profiler only reports Exceptions when it should report all Exceptions that inherit from BaseExceptions. This is ultimately reworked in a PR incorporating a simplified memory profiler module into the codebase that fixes not only this issue but also gives the possibility of getting measurements for failing tests. This workaround uses return values to work around the issue. This way pytest-monitor can check for BaseExceptions and act accordingly. Like described earlier the ultimate goal should probably be to replace the whole memory profiler as proposed in this PR: CFMTech#82
lhpt2
pushed a commit
to einhundert/pytest-monitor
that referenced
this pull request
Jul 15, 2024
The memory profiler only reports Exceptions when it should report all Exceptions that inherit from BaseExceptions. This is ultimately reworked in a PR incorporating a simplified memory profiler module into the codebase that fixes not only this issue but also gives the possibility of getting measurements for failing tests. This workaround uses return values to work around the issue. This way pytest-monitor can check for BaseExceptions and act accordingly. Like described earlier the ultimate goal should probably be to replace the whole memory profiler as proposed in this PR: CFMTech#82
7 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes issue #65 .
DEPENDS ON: #79 (#80), #78
Several people asked for support of pytest-monitor also monitoring failing tests. This PR includes the necessary changes for this feature.
Changes made in overview:
TEST_METRICSwith a columnTEST_PASSEDto make failing and passing test logs in database distinguishableThe memory profiler change
In order to get the memory usage of failing functions, the memory_profiler module had to be changed. The first approach was to make a PR on the memory_profiler project, this was marked as unmaintained though.
My options were to
I went with the second option in order to avoid big dependencies like Xray which furthermore would have needed socket based communication. This is why I introduced a new module profiler.py into the codebase being responsible for memory profiling (by making use of Process class and psutils).
Type of change
Checklist:
- [ ] Any dependent changes have been merged and published in downstream modulescreate one !)
Status: Code Review Needed,Business: Test NeededorStatus: In Progressif you are still working on the PR)@js-dieu I decided to propose my changes independent on what the original plans were, maybe it is of help, maybe not. However, this is my take on the issue.