Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speedup page class attributes accesses with shared lru_cache #1528

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 24 additions & 2 deletions volatility3/framework/symbols/linux/extensions/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2525,7 +2525,28 @@ def i_pages(self):


class page(objects.StructType):
@functools.cached_property
def __eq__(self, other):
"""Share @property's in a "vmlinux->[*pages]" lru_cache,
as thousands of page objects can be instantiated in
a single plugin run. Observed benchmarks noted
30x faster _intel_vmemmap_start accesses after first call.
The properties are in fact kernel-wide constants, and do not need
to be re-calculated on each instantiation.
Doc: https://stackoverflow.com/a/69780424

Use functools.cached_property to ignore this behaviour, as it was
observed to not call __hash__ and __eq__.
"""
return isinstance(other, page) and self.__hash__() == other.__hash__()

# Use the vmlinux to identify cache pools.
def __hash__(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redefining these is bad (see main comments).

return linux.LinuxUtilities.get_module_from_volobj_type(
self._context, self
).__hash__()

@property
@functools.lru_cache
def pageflags_enum(self) -> Dict:
"""Returns 'pageflags' enumeration key/values

Expand All @@ -2545,7 +2566,8 @@ def pageflags_enum(self) -> Dict:

return pageflags_enum

@functools.cached_property
@property
@functools.lru_cache
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method doesn't need to be a property at all, it only uses self to find the kernel, if anything this should be a LinuxUtility that takes in a kernel, or live somewhere else, there's no benefit to it living here.

def _intel_vmemmap_start(self) -> int:
"""Determine the start of the struct page array, for Intel systems.

Expand Down
Loading