pravega test

[366931]

Ask your question away

I wrote classes to extend Pravega's secondary storage in order to use other cloud services, but now I don't know how to test whether these classes are successful. Do I need to write a test class, and what should happen if the test class runs successfully?

[RaulGracia]

Hi @366931,

I think that the simplest approach would be just to mimic the existing long-term storage bindings available: https://github.com/pravega/pravega/tree/master/bindings/src/main/java/io/pravega/storage

In that package, you will find the bindings for all the storage types supported by Pravega today (ECS, S3, GCP, Azure, FS, HDFS). If you inspect the code, you will find the APIs you will need to fulfill to have a new operational storage binding. Once you have the basic functionality in place, I suggest to have a look to the test classes for the existing bindings: https://github.com/pravega/pravega/tree/master/bindings/src/test/java/io/pravega/storage

With these packages as guidelines, you should be able to develop and test a new storage binding. Once you have the tests that validate your binding alone, you can go one step forward and use your new binding in a test for the Pravega Segment Store, like this one: https://github.com/pravega/pravega/blob/0efc2f0b70aa93c65e87ffeb1878356dd145bac3/segmentstore/server/host/src/test/java/io/pravega/segmentstore/server/host/S3IntegrationTest.java

Hope it helps.

[sachin-j-joshi]

Thanks @366931 for your interest in extending Pravega LTS.

How to implement
In order to extend LTS you should implement this interface
More details about architecture/design can be found at Simplified Tier-2

• Interface to implement ChunkStorage
• For convenience you should inherit from BaseChunkStorage. The class handles most basic parameter validation and deals with converting async calls to synchronized IO calls.
• In addition, you'll need to implement StorageFactoryCreator and SimpleStorageFactory then update this manifest file
• Finally to use the storage, update this property while deploying

How to test

• We have a battery of abstract unit test classes that you can use to check concrete implementation against (Please take look at following classes SimpleStorageTests, ChunkedRollingStorageTests, ChunkStorageTests, SystemJournalTests )

• I suggest you take a look at existing providers. Please take a look at existing implementation here https://github.com/pravega/pravega/tree/master/bindings/src/test/java/io/pravega/storage . Note that we use nested classes but that is not required - it is just a coding style we chose.

• You can also implement integration tests modelled after Existing tests etc

• We have preferred to use mock implementations in our unit tests instead of actual LTS implementations (because what we care about is the protocol/interface with underlying storage and not actual implementation). But you may decide to use real implementation.

• We also suggest running existing system tests and longevity tests with your new implementation by deploying in k3s cluster. See this wiki for more details.

Hope this helps. Please do not hesitate to ask if you have more questions.

looking forward to collaborating with you on extending Pravega LTS

[yyds9292]

I'm also having issues with testing，The code I wrote
val objectPath = getObjectPath(chunkName);
long objectSize = client.getObjectMetadata(this.config.getBucket(),objectPath).getContentLength();
have an error occurred during testing