This change updates Locate operation support in the PyKMIP server,
allowing users to filter objects based on the object's
Certificate Type. Unit tests and integration tests have
been added to test and verify the correctness of this feature.
Additionally, the Locate demo scripts have also been updated to
support Certificate Type filtering. Simply use the
"--certificate-type" flag to specify a Certificate Type
enumeration values for the Locate script to filter on.
This change updates Locate operation support in the PyKMIP server,
allowing users to filter objects based on the object's
Cryptographic Usage Masks. Unit tests and integration tests have
been added to test and verify the correctness of this feature.
Additionally, the Locate demo scripts have also been updated to
support Cryptographic Usage Mask filtering. Simply use the
"--cryptographic-usage-mask" flag to specify one or more
Cryptographic Usage Mask enumeration values for the Locate script
to filter on.
This change updates Locate operation support in the PyKMIP server,
allowing users to filter objects using the offset and maximum item
constraints. The offset constraint tells the server how many
matching items should be skipped before results are returned. The
maximum items constraint tells the server how many matching items
should be returned. Unit tests and integration tests have been
added to test and verify the correctness of this feature.
Additionally, the Locate demo scripts have also been updated to
support offset and maximum item filtering. Simply use the
"--offset-items" and "--maximum-items" flags to specify offset and
maximum item values for the Locate script to filter on.
Fixes#562
This change updates the PyKMIP server's support for the Locate
operation, sorting the matched objects found by Locate by their
initial date, newest objects first. This matches the KMIP
specification's definition for how Locate results should be
ordered.
This change fixes a bug in the server engine unit tests that
check the debug logs for Locate filtering on the Initial Date
attribute. Specifically, time.asctime does not use the default
'%d' notation for stringifying numerical day values. This change
updates the string notation to match the format produced by
time.asctime.
Impacts #572
This change updates Locate operation support in the PyKMIP server,
allowing users to filter objects based on the object's Operation
Policy Name. Unit tests and integration tests have been added to
test and verify the correctness of this feature.
Additionally, the Locate demo scripts have also been updated to
support Operation Policy Name filtering. Simply use the
"--operation-policy-name" flag to specify an Operation Policy Name
string value for the Locate script to filter on.
This change updates Locate operation support in the PyKMIP server,
allowing users to filter objects based on the object's Unique
Identifier. Unit tests and integration tests have been added to
test and verify the correctness of this feature.
Additionally, the Locate demo scripts have also been updated to
support Unique Identifier filtering. Simply use the
"--unique-identifier" flag to specify a Unique Identifier string
value for the Locate script to filter on.
This change adds debug logging statements for the request and
response message encodings sent and received by the server
session. These provide direct visability into each message that
is handled by the server, facilitating debugging and correctness
checking. Given the content of these encodings may contain
sensitive information, debug logging should only be enabled when
testing or developing server features.
This change updates Locate operation support in the PyKMIP server,
allowing users to filter objects based on the object's
Cryptographic Length. If an object's type does not support the
Cryptographic Length attribute, the object is not a match. Unit
tests and integration tests have been added to test and verify
the correctness of this feature.
Additionally, the Locate demo scripts have also been updated to
support Cryptographic Length filtering. Simply use the
"--cryptographic-length" flag to specify a Cryptographic Length
integer value for the Locate script to filter on.
This change updates Locate operation support in the PyKMIP server,
allowing users to filter objects based on the object's
Cryptographic Algorithm. If an object's type does not support the
Cryptographic Algorithm attribute, that object is not a match.
Unit tests and integration tests have been added to test and
verify the correctness of this feature.
Additionally, the Locate demo scripts have also been updated to
support Cryptographic Algorithm filtering. Simply use the
"--cryptographic-algorithm" flag to specify a Cryptographic
Algorithm enumeration for the Locate script to filter on.
This change updates Locate operation support in the PyKMIP server,
allowing users to filter objects based on the object's Object Type.
Unit tests and integration tests have been added to test and verify
the correctness of this feature.
Additionally, the Locate demo scripts have also been updated to
support Object Type filtering. Simply use the "--object-type" flag
to specify an Object Type enumeration for the Locate script to
filter on.
This change updates Locate operation support in the PyKMIP server,
allowing users to filter objects based on the object's State. Unit
tests and integration tests have been added to test and verify the
correctness of this feature.
Additionally, the Locate demo scripts have also been updated to
support State filtering. Simply use the "--state" flag to specify
a State enumeration for the Locate script to filter on.
This change updates Locate operation support in the PyKMIP server,
allowing users to filter objects based on the objects InitialDate
attribute value. Specifying a single InitialDate attribute in the
Locate request will perform an exact match on objects; specifying
two InitialDate attributes will perform a ranged match. Unit tests
and integration tests have been added to test and verify the
correctness of this feature.
Additionally, the Locate demo scripts have also been updated to
support InitialDate filtering. Simply use the "--initial-date"
flag to provide one or more InitialDate values to the Locate
script to filter on those dates.
This change loosens the constraints on including attributes in
Locate messages for KMIP 2.0 clients. An early version of the KMIP
2.0 specification made attributes a required field for the Locate
payloads, breaking backwards compatibility with earlier versions
of the specification. A newer version of KMIP 2.0 reverses this
change. The library has been updated to reflect this newer version
of the specification. All relevant Locate unit tests have been
modified or removed to reflect this change.
Fixes#556
This update includes a library wide upgrade to support KMIP 2.0
for all currently supported KMIP operations. Additional changes
include documentation improvements, testing upgrades, and various
quality of life enhancements:
* Add support for Python 3.7
* Add KMIP 2.0 enumerations
* Add a new OrderedEnum subclass to handle sortable enumerations
* Add KMIP 2.0-style attribute handling
* Add utilities to convert TemplateAttributes and Attributes
* Add utilities to handle bit mask style enumerations
* Add positional argument handling for pytest calls when using tox
* Update the library documentation to include KMIP 2.0 information
* Update client exception handling / logging to simplify debugging
* Update library logging defaults to log at INFO but support DEBUG
* Update the Travis CI configuration to support Ubuntu 16.04
* Update the Travis CI configuration to output logs on failures
* Update the server to support KMIP 1.3, 1.4, and 2.0
* Update the PyKMIP clients to support changing their KMIP version
* Update server session logging for authentication failures
* Update the PyKMIP object hierarchy to propagate the KMIP version
* Update the server TLS handshake handling to avoid thread hanging
* Update the Create and Register payloads to support KMIP 2.0
* Update the Locate and CreateKeyPair payloads to support KMIP 2.0
* Update the DeriveKey / GetAttributes payloads to support KMIP 2.0
* Update the GetAttributeList / Query payloads to support KMIP 2.0
* Update attribute policy to handle KMIP 2.0 deprecated attributes
* Remove escape sequences to comply with Python 3.6 deprecations
* Fix various deprecation warnings caused by dependency upgrades
* Fix a bug decoding revocation messages for the Revoke operation
* Fix a bug specifying the function list in the Query demo script
This change fixes a bug in the server attribute handling logic
that manifests when attributes are deprecated and removed in KMIP
2.0. Now these attributes are effectively ignored for KMIP 2.0
messages, complying with the KMIP 2.0 specification.
This changes adds all of the final core updates necessary to allow
KMIP 2.0 message encoding/decoding support for the PyKMIP server.
Request and responses now dynamically adjust the KMIP version they
encode/decode under based on the KMIP version included in their
header segments. Extra server logging has also been added to show
the KMIP version specified by the client request.
Message tests have been updated to reflect these changes.
This change fixes a bug with revocation message decoding that
would cause client and server failures if the revocation message
was included in a Revoke operation call. With this fix, the client
can now send a revocation message with a Revoke request and the
revocation will occur as expected.
A ProxyKmipClient demo script for the Revoke operation has been
included to help test Revoke functionality.
Finally, an argument ordering bug with the original KMIPProxy demo
Revoke script has also been fixed.
Fixes#546
This change updates the Register payloads to properly use the new
ProtectionStorageMasks structure. Unit tests have been updated to
reflect this change.
This change updates the CreateKeyPair payloads to support the new
protection storage masks fields introduced in KMIP 2.0. The payload
unit tests have been updated to reflect these changes.
This change updates the ResponseHeader to support the new server
hashed password field added in KMIP 2.0. Unit tests have been
added to cover the change.
This change adds support for the ProtectionStorageMasks structure
which is a new addition in KMIP 2.0. A unit test suite has been
added for the new structure.
This change makes a simple renaming update to the Query response
payload, correcting how Protection Storage Masks are used and
referenced. All related unit tests have been updated to reflect
this change.
This change updates the Register payloads, adding support for
Protection Storage Masks which were added in KMIP 2.0. The
payload unit tests have been updated to reflect this change.
This change updates the Create payloads, adding support for
Protection Storage Masks which were added in KMIP 2.0. The
payload unit tests have been updated to reflect this change.
This change updates the Query payloads to support KMIP 2.0, adding
in new response components including server default information
and storage protection masks. Unit tests have been added and
updated to cover these changes.
This change upgrades the Query payloads, fixing error messages,
comments, local variables, and internal payload structure to
bring Query support up to KMIP 1.4 standards, in addition to
compliance with the current payload format. The corresponding
unit test suite has been completely rewritten to reflect these
changes.
This change prepares the Query payloads for future updates to
support KMIP 2.0.
This change adds the CapabilityInformation structure, a KMIP 1.3
addition that is used to specify details on capabilities supported
by a KMIP server. A unit test suite is included to cover the new
structure.
This change adds the ValidationInformation structure, a KMIP 1.3
addition that is used to specify details on formal validation
methods supported by a KMIP server. A unit test suite is included
to cover the new structure.
This change adds the ProfileInformation structure, a KMIP 1.3
addition that is used to specify details for supported KMIP
profiles. A unit test suite is included to cover the new
structure.
This change adds the RNGParameters structure, a KMIP 1.3 addition
that is used to specify parameters and settings associated with a
specific random number generator. A unit test suite is included
to cover the new structure.
This change adds the DefaultsInformation structure, a KMIP 2.0
addition that is used to specify default attribute values for
KMIP managed objects. A unit test suite is included to cover the
new structure.
This change adds the ObjectDefaults structure, a KMIP 2.0 addition
that is used to specify default attribute values for KMIP managed
objects. A unit test suite is included to cover the new structure.
This change updates the GetAttributeList payloads to support KMIP
2.0 features, including swapping out Attribute Names for the
Attribute Reference structure in the response payload. Unit tests
have been added to cover these changes.
This change makes minor updates to the GetAttributeList payloads,
fixing error messages, comments, and local variable names to
comply with the current payload format. The corresponding unit
test suite has been updated to reflect these changes.
This change prepares the GetAttributeList payloads for future
updates to support KMIP 2.0.
This change updates the GetAttributes payloads to support KMIP 2.0
features, including swapping out Attribute Names for the Attribute
Reference structure in the request payload and the Attribute list
for the Attributes structure in the response payload. Unit tests
have been added to cover these changes.
This change makes minor updates to the GetAttributes payloads,
fixing error messages, comments, and local variable names to
comply with the current payload format. The corresponding unit
test suite has been updated to reflect these changes.
This change prepares the GetAttributes payloads for future updates
to support KMIP 2.0.
This change adds the AttributeReference structure, a KMIP 2.0
addition that is used by several attribute-related operations.
A unit test suite is included to cover the new structure.
This change updates the DeriveKey payloads to support KMIP 2.0
features, including swapping out TemplateAttributes for the new
Attributes structure in the request payload and removing all
attribute-related encodings from the response payload. Unit tests
have been added to cover these changes.
This change makes minor updates to the DeriveKey payloads, fixing
error messages, comments, and local variable names to comply with
the current payload format. The corresponding unit test suite has
been updated to reflect these changes.
This change prepares the DeriveKey payloads for future updates to
support KMIP 2.0.
This change moves the KMIPProtocol class from the server module
to the services module. Because the client uses KMIPProtocol, and
KMIPProtocol lived in the server module, the client would end up
importing server libraries whenever it was used. If there are any
issues with server dependencies, this would cause the client to
fail for no good reason. This change now insulates the client from
the server code base and prevents this case from happening.
See #509
This change updates the CreateKeyPair payloads to support KMIP 2.0
features, including swapping out TemplateAttributes for the new
Attributes structure in the request payload and removing all
attribute-related encodings from the response payload. Unit tests
have been added to cover these changes.
This change updates the TemplateAttribute conversion utilities to
remove use of various TemplateAttribute subclasses. This reflects
the usage updates added for CreateKeyPair support. All related
unit tests have been updated to reflect this change.
This change updates the CreateKeyPair payloads to the current
payload format, adding properties for different payload attributes
and adding comparison and string operators. Changes are also made
to the PyKMIP clients and the surrounding testing infrastructure
to reflect the payload changes. The official unit test suite for
the CreateKeyPair payloads has been updated to also reflect these
changes.
This change prepares the CreateKeyPair payloads for future
updates to support KMIP 2.0.
This change updates the Locate payloads to support KMIP 2.0
features, including swapping out individual Attribute structures
for the new Attributes structure in the request payload. Unit
tests have been added to cover these changes.
This change updates the Locate payloads to the current payload
format, adding properties for different payload attributes and
adding comparison and string operators. Changes are also made to
the PyKMIP clients and the surrounding testing infrastructure to
reflect the payload changes. An official unit test suite for the
Locate payloads has also been included, which will eventually
replace the existing Locate message tests elsewhere in the test
suite.
This change prepares the Locate payloads for future updates to
support KMIP 2.0.
This changes adds several utilities for working with bit mask
enumerations, including functions to compute bit masks from lists
of enumeration values and vice versa. Unit tests have been added
to cover these new utilities.
This change updates the Register payloads to support KMIP 2.0
features, including swapping out TemplateAttributes for the new
Attributes structure in the request payload and removing all
attribute-related encodings from the response payload. Unit tests
have been added to cover these changes.
This change updates the Register payloads to the current payload
format, adding properties for different payload attributes and
adding comparison and string operators. Changes are also made to
the PyKMIP clients and the surrounding testing infrastructure to
reflect the payload changes. An official unit test suite for the
Register payloads has also been included, which will eventually
replace the existing Register message tests elsewhere in the test
suite.
This change prepares the Register payloads for future updates to
support KMIP 2.0.
When TLS handshake is performed while in `accept()` call, main thread
might blocked up to network timeout effectively locking out other
clients from being able to establish connection with PyKMIP server.
Easy way to reproduce the problem:
1. Start PyKMIP server
2. Establish TCP connection with `nc -v 127.0.0.1 5696`
3. Attempt to connect (concurrently):
`openssl s_client -host 127.0.0.1 -port 5696`
Without the fix, `openssl` would be blocked (won't even do initial TLS
handshake) until `nc` connection times out.
This change updates the Create payloads to support KMIP 2.0
features, including swapping out TemplateAttributes for the new
Attributes structure in the request payload and removing all
attribute-related encodings from the response payload. Unit tests
have been added to cover these changes.
This change adds several attribute-related utilities that support
converting TemplateAttribute structures to the new Attributes
structure and back. These utilities will be used in the updated
operation payloads to support seamless KMIP 2.0 encodings without
requiring broad payload internal and external usage changes.
Unit tests have been included to cover the new utility functions.
This change updates the Create payloads to the current payload
format, adding properties for different payload attributes and
adding comparison and string operators. Changes are also made to
the PyKMIP clients and the surrounding testing infrastructure to
reflect the payload changes. An official unit test suite for the
Create payloads has also been included, which will eventually
replace the existing Create message tests elsewhere in the test
suite.
This change prepares the Create payloads for future updates to
support KMIP 2.0.
This change adds a check to the read and write methods of the new
Attributes object that raises a new VersionNotSupported exception
if KMIP 2.0 is not the version used for encoding and decoding.
The Attributes object is not defined for older versions of KMIP
and therefore cannot be correctly encoded or decoded in those use
cases.
This change adds a new Enum subclass, OrderedEnum, that provides
comparison operators allowing for comparisons between enum values.
This will be used going forward with the KMIPVersion enum enabling
version checking on supported or unsupported object types.
This change adds a new Attributes object to the object hierarchy,
which replaces TemplateAttributes in KMIP 2.0. The old attribute
components, like the AttributeName and AttributeIndex, are no
longer used and are instead replaced with the KMIP TTLV tag for
the attributes in question. This brings the attribute encoding
process in line with the rest of the KMIP specification.
To support this change, additional attribute and enumeration
utility functions have been added to simply attribute building
and attribute/enumeration validity checking. New test cases
covering this new functionality are also included.
This change updates the PyKMIP object hierarchy's read/write
method signatures to support propagation of the KMIP version. The
introduction of KMIP 2.0 introduces future KMIP message encodings
that break backwards compatibility; to support this, PyKMIP must
know what KMIP version is being used when encoding or decoding an
object; the KMIP version residing in the client or server alone
is now insufficient. Prior versions of KMIP, namely 1.0 - 1.4,
have been backwards compatible, obviating the need for the KMIP
version at encode/decode time. Going forward, this is no longer
true.
The PyKMIP client and server have been updated to include the
KMIP version when making calls to read/write, as have the
associated test cases covering this functionality.
This change fixes a bug in the KMIPProxy client's support for the
Rekey operation. Specifically, if the operation fails and does not
return a payload, the client will still try to reference the
payload object when checking for TemplateAttribute data. This
causes an AttributeError since the payload is None. This change
fixes this and adds a unit test that covers this specific case.
Fixes#474
This change updates the PyKMIP clients, adding support for getting
and setting the KMIP version they use when making KMIP requests.
You can now do:
>>> client.kmip_version
to get the KMIP version enumeration the client is using. Use:
>>> client.kmip_version = enums.KMIPVersion.KMIP_1_1
to set the KMIP version the client uses.
The client unit tests have been updated to check and cover these
changes.
Fixes#470
This change updates the list of KMIP versions supported by the
server. While the server does not support any specific KMIP 1.3
or 1.4 features, the protocol formats are compatible across KMIP
1.0 to 1.4. Without this change, KMIP 1.3 and 1.4 requests for
older operations, like Create, Get, and Destroy, would fail.
This change also updates the server unit tests impacted by this
change.
Closes#451
This change fixes various pending deprecation warnings throughout
the library caused by recent updates to different dependencies.
While PyKMIP no longer directly triggers these warnings, some
dependencies still do when run through the test suite.
A recent style update to Python 3.6 adds deprecation W605, which
tightens the usage of invalid escape sequences. This patch removes
any instances of invalid escape sequences from the PyKMIP code
base, bringing the library back up to compliance with Python style.
As an application developer, you might expect to be able to turn on
debug logging at the root logger with something like
logging.basicConfig(level=logging.DEBUG)
However, if the application needed to fetch any secrets from a KMIP
server, these previously would be logged as part of the wire protocol.
Further, any passwords in configs would also get logged at DEBUG.
Applications would need to proactively silence such logging, as in
https://github.com/openstack/swift/commit/12b6d46
Now, we will default the logger level to INFO to suppress the debug
logging. However, seeing the on-wire data may still be useful, for
example when developing a new KMIP server. So, allow developers to
consciously set the logger level to DEBUG.
The old text made perfect sense when in a server context, trying to
read requests from clients, but KMIPProtocol is also used by *clients*
to read *responses*. Let's change it to something a little more
request/response agnostic.
Otherwise, you don't get much insight into why you had to settle for
your third configured host. Now, you can get information like
An error occurred while connecting to appliance foo.bar:
[Errno -5] No address associated with hostname
An error occurred while connecting to appliance localhost:
[Errno 111] Connection refused
even when we ultimately succeed in creating a client.
At that point, it's up to the caller to decide whether a stack trace is
appropriate; if the caller decides the connection error is recoverable,
us logging a traceback will only confuse things.
Also, prevent a TypeError during log interpolation by actually using the
argument we were providing.
Otherwise, you can hit errors with tracebacks like
Traceback (most recent call last):
...
File ".../kmip/pie/client.py", line 1573, in __enter__
self.open()
File ".../kmip/pie/client.py", line 135, in open
raise e
IOError: [Errno 2] No such file or directory
... which isn't terribly useful; it doesn't give you any information
about *what* file wasn't found. By using a bare `raise`, you preserve
the rest of the stack and get
Traceback (most recent call last):
...
File ".../kmip/pie/client.py", line 1573, in __enter__
self.open()
File ".../kmip/pie/client.py", line 131, in open
self.proxy.open()
File ".../kmip/services/kmip_client.py", line 221, in open
self._create_socket(sock)
File ".../kmip/services/kmip_client.py", line 246, in _create_socket
suppress_ragged_eofs=self.suppress_ragged_eofs)
File ".../eventlet/green/ssl.py", line 379, in wrap_socket
return GreenSSLSocket(sock, *a, **kw)
File ".../eventlet/green/ssl.py", line 68, in __init__
ca_certs, do_handshake_on_connect and six.PY2, *args, **kw)
File ".../ssl.py", line 558, in __init__
self._context.load_verify_locations(ca_certs)
IOError: [Errno 2] No such file or directory
... which makes it clear that it was a problem with the CA certificate
bundle.
This update includes new operations and features for the client
and server, in addition to other minor updates and bug fixes:
* Add Sphinx-based client and server library documentation
* Add server support for third-party authentication systems
* Add client support for the Check operation
* Add client support for the Rekey operation
* Add client support for attestation credentials
* Add functional tests for server auth. and access control
* Add payloads for the Archive and Cancel operations
* Add payloads for the GetUsageAllocation and Poll operations
* Add payloads for the ObtainLease and Recover operations
* Update the server to support group-based operation policies
* Update the server to support live loading of policy files
* Update the server to support custom database file paths
* Update the server to raise PermissionDenied on violations
* Update the client to support custom configuration file paths
* Update the ProxyKmipClient to support Registering names
* Update the ProxyKmipClient to set usage masks for Derived keys
* Update the README to reference the new documentation
* Update the Travis CI config to test building the docs
* Update the Travis CI config to run integration tests
* Remove support for Python 3.3
* Fix a DOS bug by setting the server socket timeout
* Fix a ProxyKmipClient bug with cryptographic parameter handling
* Fix a ProxyKmipClient bug with usage mask processing
This change changes the error the server uses when access control
violations occur, specifically swapping from the more agnostic
ItemNotFound to the more explicit PermissionDenied. This change
better conforms with the expected behavior of a KMIP server.
This change tweaks the format of operation policy files, renaming
the 'default' section of each policy to 'preset'. This reinforces
the idea that this section of the policy is used only when group-
based access control is disabled. It also removes any ambiguity
between this section of the policy and the actual 'default'
policy built into the server.
This change adds a basic error message that gets raised when
attempting to read an unparseable attribute (i.e., an attribute
that is not supported by the library).
Fixes#429
This change adds a new integration test suite, named 'functional',
that is specifically intended to test third-party authentication
and group-based access control with the PyKMIP server. A new tox
environment is added to handle running these tests separately from
the existing 'integration' test suite. New Travis CI configuration
and setup files have also been added to facilitate running these
tests automatically.
This change fixes a potential denial-of-service bug with the
server, setting a default timeout for all server sockets. This
allows the server to drop hung connections without blocking
forever. The interrupt triggered during accept calls is expected
and is now handled appropriately. Server unit tests have been
updated to reflect this change.
Closes#430
This change adds client support for setting custom configuration
file paths, allowing users and developers to place their settings
file wherever they want. All client demo scripts have been updated
to support a '-s' flag that can be used to set the configuration
file for the demo. Client unit tests have also been updated to
include this change.
oleksiys
Peter Hamilton
Andrey Smirnov
Peter Hamilton
Tim Burke