-
Notifications
You must be signed in to change notification settings - Fork 182
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
modules/zstd: Add initial ZSTD Encoder procs #1846
Draft
lpawelcz
wants to merge
46
commits into
google:main
Choose a base branch
from
antmicro:zstd_encoder
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- XLSStruct for easier handling and serializing/deserializing XLS structs - XLSChannel that serves as a dummy receiving channel - XLSMonitor that monitors transactions on an XLS channel - XLSDriver that can send data on an XLS channel - LatencyScoreboard that can measure latency between corresponding transactions on input and output buses - File-backed AXI memory python model Internal-tag: [#64075] Signed-off-by: Krzysztof Obłonczek <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Co-authred-by: Pawel Czarnecki <[email protected]> Co-authred-by: Robert Winkler <[email protected]> Signed-off-by: Michal Czyz <[email protected]> Signed-off-by: Pawel Czarnecki <[email protected]> Signed-off-by: Robert Winkler <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Maciej Torhan <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Maciej Torhan <[email protected]>
Remove references to buffer structs as those are not used anywhere Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Co-authored-by: Pawel Czarnecki <[email protected]> Co-authored-by: Robert Winkler <[email protected]> Signed-off-by: Maciej Torhan <[email protected]> Signed-off-by: Pawel Czarnecki <[email protected]> Signed-off-by: Robert Winkler <[email protected]>
Signed-off-by: Krzysztof Oblonczek <[email protected]>
Signed-off-by: Maciej Torhan <[email protected]>
Co-authored-by: Pawel Czarnecki <[email protected]> Signed-off-by: Maciej Torhan <[email protected]> Signed-off-by: Pawel Czarnecki <[email protected]>
Co-authored-by: Pawel Czarnecki <[email protected]> Co-authored-by: Robert Winkler <[email protected]> Signed-off-by: Maciej Torhan <[email protected]> Signed-off-by: Pawel Czarnecki <[email protected]> Signed-off-by: Robert Winkler <[email protected]>
Co-authored-by: Pawel Czarnecki <[email protected]> Signed-off-by: Robert Winkler <[email protected]> Signed-off-by: Pawel Czarnecki <[email protected]>
This reverts commit 04ad379225b706ddf492d440c673e77348d7a409.
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Internal-tag: [#67096] Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Maciej Torhan <[email protected]> Signed-off-by: Krzysztof Obłonczek <[email protected]>
Signed-off-by: Krzysztof Obłonczek <[email protected]>
Internal-tag: [#66955] Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
* Fix byte ordering when receiving a series of non-empty packets * Adjust MemReader DSLX tests Internal-tag: [#67272] Signed-off-by: Pawel Czarnecki <[email protected]>
…es function into a separate proc * Extract the operation of removing not-strobed bytes from input frames to a separate proc * Extract control logic to AxiStreamRemoveEmptyInternal proc * Optimize strobe calculation Internal-tag: [#67272] Signed-off-by: Pawel Czarnecki <[email protected]>
Fix paramaterization of the proc Internal-tag: [#67272] Signed-off-by: Pawel Czarnecki <[email protected]>
…ackets Add AxiStreamRemoveEmpty proc to the processing pipeline. It removes non-strobed bytes from the input AXI Stream frames and forms full frames (ensures that only the last input data packet won't be full). Internal-tag: [#67272] Signed-off-by: Pawel Czarnecki <[email protected]>
* Extract control logic to MemWriterInternal proc * Create alias for the MemWriter response type Internal-tag: [#67272] Signed-off-by: Pawel Czarnecki <[email protected]>
SequenceExecutor: * Add output channel in the format compliant with MemWriter input data channel type ZstdDecoder: * Add MemWriter proc: * Write request formed based on the address of the OutputBuffer CSR and FrameContentSize field from the Frame Header * Data to write is sent out to the proc by the SequenceExecutor * Transition to the FINISH state (and triggers notify channel) only after receiving the response from the MemWriter * DSLX tests: * Receive decoded data sent out on the AXI interface by the MemWriter proc * Mock the output memory buffer as a DSLX array * Cocotb tests: * Move third-party verilog modules (AXI Interconnect) to external directory * Replace AXI Interconnect with AXI Crossbar that handles simultaneous AXI Read and Write transactions * Add reference memory and fill it with expected data for comparison against testbench memory at the end of the decoding Internal-tag: [#67272] Signed-off-by: Pawel Czarnecki <[email protected]>
* Remove Repacketizer proc * Remove stream-based output channels from * SequenceExecutor * ZstdDecoder Internal-tag: [#67272] Signed-off-by: Pawel Czarnecki <[email protected]>
* Decode multiple ZSTD frames in a single cocotb testbench * Add one cocotb testbench per type of the ZSTD frames: * Frames with RAW blocks only * Frames with RLE blocks only * Frames with Compressed blocks only (disabled) * Frames with mixed blocks (disabled) Internal-tag: [#67272] Signed-off-by: Pawel Czarnecki <[email protected]>
Internal-tag: [#67272] Signed-off-by: Pawel Czarnecki <[email protected]>
…on for verilog library Signed-off-by: Pawel Czarnecki <[email protected]>
…st cases Caused by timeouts after rebase - looks like regression in the performance of codegen Signed-off-by: Pawel Czarnecki <[email protected]>
…atus enums Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Maciej Torhan <[email protected]>
Signed-off-by: Maciej Torhan <[email protected]>
Signed-off-by: Robert Winkler <[email protected]> Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Krzysztof Oblonczek <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds initial procs of the ZSTD Encoder. It's based on #1654.
Changes in this PR consist of:
AlignedParallelRam
proc - Responsible for handling Read/Write operations on multiple RAM models in parallel.HashTable
proc - Generic Hash Table implementation. It uses underlying RAM models for operation and is controlled with a Read/Write Request/Response channels interface. In the ZSTD Encoder, it is used for fast lookup of repeating symbols in the input data buffer.HistoryBuffer
proc - Data buffer capable of storing historical data and outputting said data from arbitrary chosen offset up to the top of the buffer. UsesAlignedParallelRam
proc underneath.MatchFinder
proc - Processes data stream from the input buffer in the system memory. Searches for repeating symbol patterns in theHashTable
proc. Symbols that are not found are stored directly as literals in the literals buffer, while those that have matches are stored as sequences in the sequence buffer, using the matched offset and length information. Sequence and Literals Buffers are implemented in the system memory accessible via the AXI interface.This PR is currently a
WIP
.The following work needs to be finished to mark this PR as
ready for review
:MatchFinder
codegen