Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SourceRecordReader implementations should use milliseconds to create their records #15

Closed
ahawtho opened this issue Oct 6, 2022 · 3 comments
Closed

SourceRecordReader implementations should use milliseconds to create their records #15

ahawtho opened this issue Oct 6, 2022 · 3 comments
Labels
released Issue has been released

Comments

@ahawtho
Copy link
Contributor

ahawtho commented Oct 6, 2022

Kafka record timestamps are defined in milliseconds, but the StreamSourceRecordReader class and the KeySourceRecordReader class use the following to create their records:

		return new SourceRecord(..., Instant.now().getEpochSecond());

These should be switched to Instant.now().toEpochMilli().

The effect is that when the source connector appends a message to a topic, if the topic has retention.ms set to a reasonable value, the messages are immediately deleted from the topic.

This can be worked around by setting the topic configuration value:

message.timestamp.type: LogAppendTime
ahawtho added a commit to coursehero/redis-kafka-connect that referenced this issue Oct 6, 2022
@ahawtho
Fixes redis-field-engineering#15
d75498e 
Kafka message timestamps are in milliseconds, but the SourceRecordReader
implementations in this package were using seconds. This just changes
them to use milliseconds instead.
ahawtho added a commit to coursehero/redis-kafka-connect that referenced this issue Oct 7, 2022
@ahawtho
Adds tests
430e0ab 
redis-field-engineering#15
@jruaux jruaux closed this as completed in f2ae363 Oct 11, 2022
jruaux added a commit that referenced this issue Oct 11, 2022
@jruaux
Merge pull request #16 from coursehero/master
029fd22 
Fixes #15
@ahawtho
Copy link
Contributor Author

ahawtho commented Oct 11, 2022

Thanks for the quick turnaround. Any idea when will this make a release?

@jruaux
Copy link
Collaborator

jruaux commented Oct 16, 2022

Thank you for your contribution. This is included in version 7.2.

@jruaux
Copy link
Collaborator

jruaux commented Jun 21, 2023

🎉 This issue has been resolved in v0.7.5 (Release Notes)

@jruaux jruaux added the released Issue has been released label Jun 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
released Issue has been released
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ahawtho @jruaux