-
Notifications
You must be signed in to change notification settings - Fork 86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixes issue with non-seekable sources breaking StreamEncryptor and StreamDecryptor handling #14
Merged
Merged
Changes from 2 commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
83d3719
adding functional test to ensure decrypt works with source streams wh…
mattsb42-aws 8620eef
turning internal.utils into a module and moving stream utils into int…
mattsb42-aws e5bd7cd
adding pytest-catchlog to test dependencies
mattsb42-aws dcc4b84
adding functional tests to ensure that all uses of encrypt/decrypt/En…
mattsb42-aws dac0ea1
moving and expanding unit tests for streams utils
mattsb42-aws fa80bf9
documenting additional tox testenvs
mattsb42-aws 3293bd2
updating StreamEncryptor and StreamDecryptor to not require seek() or…
mattsb42-aws 685827d
updating ROStream references to reflect new location in internal.util…
mattsb42-aws 1afeead
linting fixes
mattsb42-aws 8418d1a
updating version to 1.3.2 and updating changelog
mattsb42-aws bc8ff2f
updates to utils.streams to fix attribute/method naming confusion and…
mattsb42-aws c83a818
replacing PassThroughStream with wrapt.ProxyObject
mattsb42-aws a4f4df7
moving TeeStream internal tee to private name to avoid potential futu…
mattsb42-aws 7d42cf0
Merge branch 'master' into non_seekable_sources
mattsb42-aws File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
# Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"). You | ||
# may not use this file except in compliance with the License. A copy of | ||
# the License is located at | ||
# | ||
# http://aws.amazon.com/apache2.0/ | ||
# | ||
# or in the "license" file accompanying this file. This file is | ||
# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF | ||
# ANY KIND, either express or implied. See the License for the specific | ||
# language governing permissions and limitations under the License. | ||
"""Helper stream utility objects for AWS Encryption SDK.""" | ||
import io | ||
|
||
from aws_encryption_sdk.exceptions import ActionNotAllowedError | ||
|
||
|
||
class PassThroughStream(object): | ||
"""Provides a pass-through interface on top of a stream object. | ||
|
||
:param source_stream: File-like object | ||
""" | ||
|
||
def __init__(self, source_stream): | ||
"""Prepares the passthroughs.""" | ||
self._source_stream = source_stream | ||
self._duplicate_api() | ||
|
||
def _duplicate_api(self): | ||
"""Maps the source stream API onto this object.""" | ||
source_attributes = set([ | ||
method for method in dir(self._source_stream) | ||
if not method.startswith('_') | ||
]) | ||
self_attributes = set(dir(self)) | ||
for attribute in source_attributes.difference(self_attributes): | ||
setattr(self, attribute, getattr(self._source_stream, attribute)) | ||
|
||
|
||
class ROStream(PassThroughStream): | ||
"""Provides a read-only interface on top of a stream object. | ||
|
||
Used to provide MasterKeyProviders with read-only access to plaintext. | ||
|
||
:param source_stream: File-like object | ||
""" | ||
|
||
def write(self, b): # pylint: disable=unused-argument | ||
"""Blocks calls to write. | ||
|
||
:raises ActionNotAllowedError: when called | ||
""" | ||
raise ActionNotAllowedError('Write not allowed on ROStream objects') | ||
|
||
|
||
class TeeStream(PassThroughStream): | ||
"""Provides a ``tee``-like interface on top of a stream object, which collects read bytes | ||
into a local :class:`io.BytesIO`. | ||
|
||
:param source_stream: File-like object | ||
""" | ||
|
||
def __init__(self, source_stream): | ||
"""Creates the local tee stream.""" | ||
self.tee = io.BytesIO() | ||
super(TeeStream, self).__init__(source_stream) | ||
|
||
def read(self, b=None): | ||
"""Reads data from source, copying it into ``tee`` before returning. | ||
|
||
:param int b: number of bytes to read | ||
""" | ||
data = self._source_stream.read(b) | ||
self.tee.write(data) | ||
return data |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might just be that Python isn't my daily driver, but I want to ask about a few things:
attributes
on the LHS andmethod
s on the RHS, but it looks like you're getting everything not prefixed with_
, not only methodssource_stream
, would it make sense to havePassThroughStream
itself inherit from the base stream type (if such a thing practically exists) and have each of the defined stream methods delegate to the underlyingsource_stream
? Or, if not, should it instead copy everything?For example: say we get a stream object
CoolStream
that is itself customized and has a (for example)_report_bytes_()
API that is called by the local implementation ofwrite()
.PassThroughStream
copieswrite()
but not_report_bytes()
, so whenwrite()
is invoked, it unexpectedly fails. Alternatively, ifPassThroughStream
is say anIOBase
(for example; might not be the right choice) that has all the standard stream methods,PassThroughStream.write()
can callsource_stream.write()
and treat the wrapped stream as more of a black box. I think that would drop some edge cases but preserve intent/functionality here?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch on the attributes/methods. Fixed that.
The problem with just subclassing
io.BaseIO
is that these are really meant to front file-like objects, not just streams (fixed that terminology in the comments too). File-like objects (of which streams are a subset) have a consistent API but it is also quite large, so I would rather avoid manually duplicating it.Effectively what this is doing is duplicating the public API of
_source_stream
onto the instance ofPassThroughStream
in the form of pointers to the attributes on_source_stream
. This makes it so that any public interactions with the instance ofPassThroughStream
behave exactly as they would if they were made to_source_stream
directly. Behavior of calls inside_source_stream
are unaffected, as they are still happening within the context of that (unmodified) object.Aside from the obvious overrides, the only place where this should introduce any unexpected behavior is if someone is making calls to private-hinted attributes on
_source_stream
, and I think that is a safe place to draw the line, and it matches with the convention laid out in PEP8.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Per offline discussion: replacing
PassThroughStream
withProxyObject
from wrapt.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, sorry for my confusion on the API duplication, I did not remember that interaction correctly. wrapt is a good find, that turned out as a nice clean drop-in.