Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix nerdctl ps slow on heavy IO system by using goroutine #3673

Merged
merged 1 commit into from
Dec 2, 2024

Conversation

ningmingxiao
Copy link
Contributor

@ningmingxiao ningmingxiao commented Nov 13, 2024

formatter.ContainerStatus will use some time. If plenty of calls are coming, this will use much time.
before

time nerdctl_old -n k8s.io  ps -a
real    2m27.206s
user    0m0.336s
sys     0m0.218s

after this commit

time nerdctl_new -n k8s.io  ps -a
real    0m3.333s
user    0m0.145s
sys     0m0.074s

@AkihiroSuda

@ningmingxiao ningmingxiao changed the title fix nerdctl ps slow by using goroutine fix nerdctl ps slow on heavy IO system by using goroutine Nov 13, 2024
@ningmingxiao ningmingxiao force-pushed the nerd_ps branch 6 times, most recently from 3eac91f to b5d0902 Compare November 14, 2024 04:46
pkg/cmd/container/list.go Outdated Show resolved Hide resolved
pkg/cmd/container/list.go Outdated Show resolved Hide resolved
Copy link
Member

@djdongjin djdongjin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@ningmingxiao ningmingxiao force-pushed the nerd_ps branch 4 times, most recently from b629627 to 87bb294 Compare November 20, 2024 02:07
@fahedouch fahedouch self-requested a review November 20, 2024 22:15
// formatter.ContainerStatus(ctx, c) is time consuming so we do it in goroutines and return the container's id with status as a map.
// prepareContainers func will use this map to avoid call formatter.ContainerStatus again.
for _, c := range containers {
wg.Add(1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
wg.Add(1)
c := c
wg.Add(1)

the above line code simplifies the code and avoid using a mutex, right ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ningmingxiao WDYT ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

avoid using a mutex

I don't think it will avoid the mutex. Current write to a map still need the mutex.

Copy link
Member

@fahedouch fahedouch Dec 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indeed map are note safe for concurrent writes. thanks @djdongjin for your pointer ;)

pkg/cmd/container/list.go Outdated Show resolved Hide resolved
pkg/cmd/container/list.go Outdated Show resolved Hide resolved
pkg/cmd/container/list.go Outdated Show resolved Hide resolved
@fahedouch
Copy link
Member

@ningmingxiao PTAL on this thread #3673 (comment)

Copy link
Member

@fahedouch fahedouch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@AkihiroSuda AkihiroSuda added this to the v2.0.2 (tentative) milestone Dec 2, 2024
@fahedouch fahedouch merged commit 1f81225 into containerd:main Dec 2, 2024
30 checks passed
@ningmingxiao ningmingxiao deleted the nerd_ps branch December 19, 2024 01:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants