Set ulimit -n to 1024 on centos targets (#70)

rpm on centos 7 calls fcntl on every FD up to the max in order to set CLOEXEC, and the maximum number of open FDs in docker on our runners was 2**30 - 8 == 1073741816. (See rpm-software-management/rpm#444). ulimit can't be configured in a Dockerfile, and there doesn't seem to be a way to pass argument to `docker build` if you are using the default github action, so just call ulimit before the big yum install. There might be a way to configure it on our infra runner side too, I don't know.
edgedb · Dec 22, 2023 · 2c6386c · 2c6386c
1 parent f2177af
commit 2c6386c
Show file tree

Hide file tree

Showing 6 changed files with 24 additions and 9 deletions.
diff --git a/integration/linux/build/Dockerfile-centos.template b/integration/linux/build/Dockerfile-centos.template
@@ -16,7 +16,10 @@ RUN (echo precedence ::ffff:0:0/96 100 >> /etc/gai.conf \
     && sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-* \
     && yum update -y) || true)
 
-RUN yum install -y \
+# rpm on centos 7 iterates over all fds up to the limit, which is
+# extremely slow. Force it to be small.
+RUN ulimit -n 1024 \
+    && yum install -y \
     wget gcc make zlib-devel openssl-devel sqlite-devel bzip2-devel \
     expat-devel ncurses-devel gdbm-devel readline-devel libuuid-devel \
 	curl-devel xz-devel libffi-devel gettext glibc-langpack-en \

diff --git a/integration/linux/build/Dockerfile-linux.template b/integration/linux/build/Dockerfile-linux.template
@@ -20,8 +20,10 @@ ENV NODE_VERSION %%PLACEHOLDER%%
 ENV YARN_VERSION %%PLACEHOLDER%%
 ENV GCC_VERSION 11
 
-
-RUN yum install -y centos-release-scl epel-release \
+# rpm on centos 7 iterates over all fds up to the limit, which is
+# extremely slow. Force it to be small.
+RUN ulimit -n 1024 \
+        && yum install -y centos-release-scl epel-release \
 	&& yum install -y \
 	devtoolset-${GCC_VERSION} make patch flex bison \
     wget zlib-devel openssl-devel sqlite-devel bzip2 bzip2-devel \

diff --git a/integration/linux/build/centos-7/Dockerfile b/integration/linux/build/centos-7/Dockerfile
@@ -22,7 +22,10 @@ RUN (echo precedence ::ffff:0:0/96 100 >> /etc/gai.conf \
     && sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-* \
     && yum update -y) || true)
 
-RUN yum install -y \
+# rpm on centos 7 iterates over all fds up to the limit, which is
+# extremely slow. Force it to be small.
+RUN ulimit -n 1024 \
+    && yum install -y \
     wget gcc make zlib-devel openssl-devel sqlite-devel bzip2-devel \
     expat-devel ncurses-devel gdbm-devel readline-devel libuuid-devel \
 	curl-devel xz-devel libffi-devel gettext glibc-langpack-en \

diff --git a/integration/linux/build/centos-8/Dockerfile b/integration/linux/build/centos-8/Dockerfile
@@ -22,7 +22,10 @@ RUN (echo precedence ::ffff:0:0/96 100 >> /etc/gai.conf \
     && sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-* \
     && yum update -y) || true)
 
-RUN yum install -y \
+# rpm on centos 7 iterates over all fds up to the limit, which is
+# extremely slow. Force it to be small.
+RUN ulimit -n 1024 \
+    && yum install -y \
     wget gcc make zlib-devel openssl-devel sqlite-devel bzip2-devel \
     expat-devel ncurses-devel gdbm-devel readline-devel libuuid-devel \
 	curl-devel xz-devel libffi-devel gettext glibc-langpack-en \

diff --git a/integration/linux/build/linux-aarch64/Dockerfile b/integration/linux/build/linux-aarch64/Dockerfile
@@ -26,8 +26,10 @@ ENV NODE_VERSION 16.16.0
 ENV YARN_VERSION 1.22.19
 ENV GCC_VERSION 10
 
-
-RUN yum install -y centos-release-scl epel-release \
+# rpm on centos 7 iterates over all fds up to the limit, which is
+# extremely slow. Force it to be small.
+RUN ulimit -n 1024 \
+        && yum install -y centos-release-scl epel-release \
 	&& yum install -y \
 	devtoolset-${GCC_VERSION} make patch flex bison \
     wget zlib-devel openssl-devel sqlite-devel bzip2 bzip2-devel \

diff --git a/integration/linux/build/linux-x86_64/Dockerfile b/integration/linux/build/linux-x86_64/Dockerfile
@@ -26,8 +26,10 @@ ENV NODE_VERSION 16.16.0
 ENV YARN_VERSION 1.22.19
 ENV GCC_VERSION 11
 
-
-RUN yum install -y centos-release-scl epel-release \
+# rpm on centos 7 iterates over all fds up to the limit, which is
+# extremely slow. Force it to be small.
+RUN ulimit -n 1024 \
+        && yum install -y centos-release-scl epel-release \
 	&& yum install -y \
 	devtoolset-${GCC_VERSION} make patch flex bison \
     wget zlib-devel openssl-devel sqlite-devel bzip2 bzip2-devel \