Repository navigation
fix: partial write error handling - #239
NguyenHoangSon96 wants to merge 6 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR refactors write error handling to better distinguish partial-write errors from generic write failures by parsing JSON error bodies and conditionally raising InfluxDBPartialWriteError only when accept_partial is enabled and the response matches the expected “rejected rows” list format.
Changes:
- Update write exception translation to parse JSON bodies and format per-line rejection details for partial writes.
- Adjust sync REST client response decoding to normalize empty bodies to
None. - Expand and update tests to cover partial-write detection, message formatting, and fallback behaviors.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
tests/test_write_api.py |
Adds broader unit coverage for write-error classification and fallback messaging; updates existing error-translation tests. |
tests/test_influxdb_client_3_integration.py |
Loosens assertion on v3 error message content and narrows the tested accept_partial configuration. |
influxdb_client_3/write_client/client/write_api.py |
Implements new JSON parsing and partial-write detection/formatting in _translate_write_exception. |
influxdb_client_3/write_client/_sync/rest_client.py |
Avoids decoding empty bodies and normalizes empty data to None. |
influxdb_client_3/exceptions/exceptions.py |
Changes exception message handling and updates partial-write error types/constructors. |
Suppressed comments (1)
influxdb_client_3/exceptions/exceptions.py:69
- These PEP 604 union annotations (
int | None, etc.) will fail to parse on Python <3.10. UseOptional[...](withOptionalimported fromtyping) to preserve compatibility.
class InfluxDBPartialWriteLineError:
line_number: int | None
error_message: str | None
original_line: str | None
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| def __init__(self, response: HTTPResponse = None, message: str = None): | ||
| """Initialize the InfluxDBError handler.""" | ||
| if response is not None: | ||
| self.response = response | ||
| self.message = self._get_message(response) | ||
| self.message = message | ||
| self.retry_after = response.getheader('Retry-After') | ||
| else: | ||
| self.response = None | ||
| self.message = message or 'no response' | ||
| self.retry_after = None | ||
| super().__init__(self.message) |
0e798b1 to
3352e90
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Suppressed comments (2)
influxdb_client_3/exceptions/exceptions.py:5
- Python 3.9 is still in the CI matrix, but this module now uses PEP 604 union syntax elsewhere (
int | None), so you’ll needOptionalavailable for 3.9-compatible type annotations.
from typing import List
influxdb_client_3/exceptions/exceptions.py:69
int | None/str | Nonetype syntax is a Python 3.10+ feature and will raise a SyntaxError on Python 3.9 (which is still in.github/workflows/pylint.yml). UseOptional[...]instead for compatibility.
class InfluxDBPartialWriteLineError:
line_number: int | None
error_message: str | None
original_line: str | None
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #239 +/- ##
==========================================
+ Coverage 86.34% 87.65% +1.31%
==========================================
Files 28 28
Lines 2072 2042 -30
==========================================
+ Hits 1789 1790 +1
+ Misses 283 252 -31 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
1ea4183 to
1c5111d
Compare
28581e6 to
4f769ef
Compare
bednar
left a comment
There was a problem hiding this comment.
I found five issues that need to be addressed before approval.
dc587df to
90321e4
Compare
dc4f14d to
98610a0
Compare
d4c9bd5 to
8b146c8
Compare
8b146c8 to
6cf2352
Compare
|
@karel-rehor |
karel-rehor
left a comment
There was a problem hiding this comment.
I have some concerns about missed opportunities to improve and simplify the API. This is of course open for discussion.
|
|
||
|
|
||
| # This error is for all write operations | ||
| class InfluxDBError(InfluxDB3ClientError): |
There was a problem hiding this comment.
This probably should have been caught in PR140, but to be fully self descriptive and to be more in line with other client APIs, since this error is only used with writes, perhaps it should be name InfluxDBWriteError or InfluxDBServerError or InfluxdDBRestCallError.
| self.response = None | ||
| self.message = message or 'no response' | ||
| self.retry_after = None | ||
| super().__init__(self.message) |
There was a problem hiding this comment.
Agree with Copilot. Need a fallback message when argument is None.
There was a problem hiding this comment.
Seeing that we now have InfluxDBError added in the file exceptions.py can we refactor some technical debt here?
I would perhaps consider the following.
- move
write_exceptionsto theexceptionsdirectory. - remove
ApiExceptionand merge it with what is currentlyInfluxDBError, which works more likeInfluxDBWriteError(see code comment, and other PR comment).- Consider that the API at this stage is getting needlessly complex. For an API exception we now have an inheritance chain of
ApiException -> InfluxDBError -> InfluxDB3ClientError -> Exception - I see that we want to try and preserve Jay Clifford's work, but a lot of this is a direct transposition from Influxdb2 and this PR is an opportunity to further simplify the API as much as possible.
- Consider that the API at this stage is getting needlessly complex. For an API exception we now have an inheritance chain of
- Move any exception or error classes related to the write API into
write_exceptions.py. - ensure naming of all exception classes and files within the API uses consistently either
<Tokens>Exceptionor<Tokens>Errorbut not both. - Some global functions here might make more sense as static class methods, but this requires futher reflection.
| exc: ApiException, | ||
| use_v2_api=False, | ||
| accept_partial=False, | ||
| ) -> Union[ApiException, InfluxDBPartialWriteError]: |
There was a problem hiding this comment.
If ApiException were to be refactored to simplify inheritence a Union might not be required here and if python OOP works like other languages, I would expect any class inheriting from InfluxDBError could be returned.
Closes #
Proposed Changes
The current exception classes hierarchy are
InfluxDBError->InfluxDBPartialWriteErrorandApiExceptionInfluxDBErrorthere is an important function_get_message(self, response), this function will run everytime a subclass ofInfluxDBErroris initialized, inside this function there is a quite heavy function will run, that is_parse_partial_write_line_error_info(data), so when a partial write error occurs_get_message(self, response)will be called at least twice.Some issues:
InfluxDBErrorwill have the same way of parsing the error messages like we do in_get_message(self, response)function is not correct... I think.We can try to make everything work more efficiently as possible with the current exception classes implementation, but I still feel It very wrong on the architecture and design perspective.
What I'm trying to do right now is moving all logic inside exception classes to WriteApi class (or wherever than inside exception classes) and making them as lightweight as possible. They should only carry information about the errors;
Checklist