-
Notifications
You must be signed in to change notification settings - Fork 126
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace func trie with hashmap #179
base: master
Are you sure you want to change the base?
Conversation
@visitorckw, can you comment this? |
Looks good as is. However, as mentioned, a large number of functions may cause excessive collisions and slow down performance. For smaller function counts, the default 512 buckets might be overkill. Therefore, a radix tree with dynamic memory allocation could still be a method worth exploring in the future. |
I'm concerning that dynamic memory allocation at this moment is not reliable and potentially flawed, I've attempted to implement rehashing algorithm before, but on stage 2 the compilation will fail, while the GCC and stage 1 are fine. |
1f5ea61
to
0c48824
Compare
Refine the message "This also enables us to refactor more structures later with hashmap implementation in shecc." as following:
You don't have to mention the project name unless it does impact something you are referring. |
0c48824
to
400e3ae
Compare
1bacbd7
to
cb82f7a
Compare
Previously, trie implementation is not consistent, mainly because of using index to point the referencing func_t to FUNCS, additionally, trie's advantage is that enables prefix lookup, but in shecc, it hasn't been used in this way, furthur more, it takes 512 bytes per trie node, while in this implementation, it 24 + W (W stands for key length including NULL character) bytes per hashmap bucket node, which significantly reduces memory usage. This also allows for future refactoring of additional structures using a hashmap implementation.
cb82f7a
to
4a5192e
Compare
|
||
for (; *key; key++) { | ||
hash ^= *key; | ||
hash *= 0x01000193; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The multiplication here may cause overflow, leading to undefined behavior. Signed integer overflow is undefined, while unsigned integer overflow is not. Since shecc currently lacks support for unsigned integers, we might consider adding it to address this issue.
Previously, trie implementation is not consistent, mainly because of using index to point the referencing
func_t
toFUNCS
, additionally, it lacks of dynamic allocation which might cause segmentation fault and results more technical debt to debug on eitherFUNCS
orFUNCS_TRIE
. Thus, in this PR, we can resolve this issue by introducing a dynamic hashmap.Current implementation is using FNV-1a hashing algorithm (32-bit edition to be precise), and due to lack of unsigned integer implementation, hashing result ranges from
0
to2,147,483,647
.Notice that current implementation may suffer from lookup issue when the function amount keeps increasing since current hashmap implementation doesn't offer rehashing based on load factor (which ideally, 0.75 would be best and currently shecc does not support floating number).
This also enables us to refactor more structures later with hashmap implementation in shecc.
Benchmark for ./tests/hello.c compilation
Before
After