Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pagination or lazy-load to results table #39

Closed
ITAYC0HEN opened this issue Apr 4, 2020 · 4 comments
Closed

Add pagination or lazy-load to results table #39

ITAYC0HEN opened this issue Apr 4, 2020 · 4 comments
Assignees
Labels
priority:high Priority: high
Milestone

Comments

@ITAYC0HEN
Copy link
Collaborator

Description
When executing a query with many results, the interface will show all the matches in the results table. The table can be with tens of thousands of results for a big dataset, which will make the page slow to response. I suggest implementing pagination (the xhr requests are already fetching 50 at a time) or a lazy-loading

@msm-code
Copy link
Contributor

msm-code commented Apr 4, 2020

Yeah, that can be quite a pain

@msm-code msm-code added this to the v1.1.0 milestone Apr 8, 2020
@msm-code
Copy link
Contributor

To test:

  1. Fenerate a few test files:
import string
import os
import random

for i in range(40000):
    print(i)
    data = os.urandom(40000)
    open(os.urandom(16).hex(), 'wb').write(data)
  1. index them all

  2. Do some trivial query like:

rule hmm {
strings: $a = "a"
condition: $a
}
  1. Observe that the query finishes quickly, but loading all the results takes ages.

@ITAYC0HEN
Copy link
Collaborator Author

#78 is somewhat related to the planning of the new table. A modern approach to a filterable table.

@msm-code
Copy link
Contributor

fixed with #96

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority:high Priority: high
Projects
None yet
Development

No branches or pull requests

3 participants