Skip to content

feat: Implement AckSetTracker which tracks message acknowledgements.#19

Merged
dpcollins-google merged 2 commits into
masterfrom
ack_set_tracker
Sep 22, 2020
Merged

feat: Implement AckSetTracker which tracks message acknowledgements.#19
dpcollins-google merged 2 commits into
masterfrom
ack_set_tracker

Conversation

@dpcollins-google

Copy link
Copy Markdown
Contributor

Note that it is awkward to structure this like the java version, as there is no "AsyncCallable" type in python.

Note that it is awkward to structure this like the java version, as there is no "AsyncCallable" type in python.
@dpcollins-google dpcollins-google requested a review from a team September 18, 2020 14:16
@google-cla google-cla Bot added the cla: yes This human has signed the Contributor License Agreement. label Sep 18, 2020
self._acks = queue.PriorityQueue()

def track(self, offset: int):
if len(self._receipts) > 0:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This leaves the AckTracker in an inconsistent state on exceptions. Also, there is no need to pop and push, you can just access the first element with [0].

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to pop and push

Done.

leaves the AckTracker in an inconsistent state on exceptions

Not quite sure what you mean here, can you elaborate?

async def ack(self, offset: int):
self._acks.put_nowait(offset)
prefix_acked_offset: Optional[int] = None
while len(self._receipts) != 0 and not self._acks.empty():

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you peek the queues instead of popping and pushing?

PriorityQueue.queue[index] and deque[index].

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can, but this seems super implicit as it relies on 1) the fact that Priorityqueue class has a queue submember (undocumented) and 2) the PriorityQueue class uses the heapq module and the element at index 0 of a queue managed by that module is the minimum one. I don't like that, but can make this change if you prefer.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think that's fair. Let's leave as is.

@@ -0,0 +1,46 @@
from asynctest.mock import MagicMock, CoroutineMock, call

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CoroutineMock is unused, unless it does something to the global state?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment thread google/cloud/pubsublite/cloudpubsub/internal/ack_set_tracker.py Outdated
@abstractmethod
def track(self, offset: int):
"""
Track the provided offset.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: leave a comment in the interface saying that these should be strictly increasing? It would be nice to include the basic behavior on the class description; i.e. that it commits contiguous ranges.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment thread google/cloud/pubsublite/cloudpubsub/internal/ack_set_tracker.py
async def ack(self, offset: int):
self._acks.put_nowait(offset)
prefix_acked_offset: Optional[int] = None
while len(self._receipts) != 0 and not self._acks.empty():

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think that's fair. Let's leave as is.

@dpcollins-google dpcollins-google merged commit 7f88458 into master Sep 22, 2020
@anguillanneuf anguillanneuf deleted the ack_set_tracker branch March 25, 2022 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants