ARG_MAX, maximum length of arguments for a new process

command: arg list too long 

2002-06-06 .. 2022-06-27 (see recent changes)

Here you'll find

• More about the nature of this limit
• The effectively usable space ...and a way to determine it reliably
• ...alternatively: about the GNU autoconf check
• How to avoid the limit in a shell
• other limits: number of arguments and maximum length of one argument
• Actual values for ARG_MAX (with some more details in the footnotes)
• More pages

More about the nature of this limit

You will see the error message "arg list too long " if you try to call a program with too many arguments, that is,
most likely in connection with pattern matching:

    $ command * 

On some systems the limit is even hit with "grep pattern /usr/include/*/*" (using find or a recursive grep would be more appropriate anyway).

It's only the exec() system call and its direct variants, which will yield this error.
They return the corresponding error condition E2BIG (<sys/errno.h>).

The shell is not to blame, it just delivers this error to you.
In fact, shell expansion is not a problem, because here exec() is not needed, yet.
Expansion is only limited by the virtual memory system resources ^[1].
Thus the following commands work smoothly, because instead of handing over too many arguments to a new process,
they only make use of a shell built-in (echo) or iterate over the arguments with a control structure (for loop):

    /dir-with-many-files$ echo * | wc -c
    /dir-with-many-files$ for i in * ; do grep ARG_MAX "$i"; done

There are different ways to learn the upper limit (not the actual limit in effect in your current environemt, which might be lower)

• command: getconf ARG_MAX ^[2]
• system call: sysconf(_SC_ARG_MAX) ^[3]
• system header: ARG_MAX in e.g. <[sys/]limits.h> ^[4]
• try xargs --show-limits^[5], if you use GNU xargs

(However, on the few system that have no limit for ARG_MAX, these methods wrongly might print a limit.)

From Version 7 on the limit was defined by NCARGS (usually in <[sys/]params.h>),
Later, ARG_MAX was introduced with 4.4BSD and System V.

In contrast to the headers, sysconf and getconf tell the limit which is actually in effect.
This is relevant on systems which allow to change it at run time (AIX), by reconfiguration (UnixWare, IRIX),
by recompiling (e.g. Linux) or by applying patches (HP-UX 10) - see the end notes for more details.
(Usually these are solutions for special requirements only, because increasing the limit doesn't solve the problem.)

#include <stdio.h>
#include <unistd.h>
int main() {
    return printf("ARG_MAX: %ld\n", sysconf(_SC_ARG_MAX));
} 

The effectively usable space

When looking at ARG_MAX/NCARGS, you have to consider the space comsumption by both argv[] and envp[] (arguments and environment).
Thus you have to decrease ARG_MAX at least by the results of "env|wc -c" and "env|wc -l * 4" ^[5] for a good estimation of the currently available space.

POSIX suggests to subtract 2048 additionally so that the process may savely modify its environment. A quick estimation with the getconf command:
(all the calculations inspired by a post from Gunnar Ritter in de.comp.os.unix.shell, <3B70A6AD.3L8115910@bigfoot.de>)

 expr `getconf ARG_MAX` - `env|wc -c` - `env|wc -l` \* 4 - 2048

...and a way to determine the limit reliably

...alternatively: about the GNU autoconf check

In a loop with increasing n, the check tries an exec() with an argument length of 2ⁿ (but won't check for n higher than 16, that is 512kB).
The maximum is ARG_MAX/2 if ARG_MAX is a power of 2.
Finally, the found value is divided by 2 (for safety), with the reason "C++ compilers can tack on massive amounts of additional arguments".

How to avoid the limit in a shell

• iterate with the shell:
  • for i in *; do command "$i"; done     (simple, completely robust and portable, may be very slow)    
  • printf '%s\0' *|xargs -0 command     (works only if printf is a built-in, but then it can be much faster on high counts. thanks to Michael Klement)

• iterate with find
  • find . -exec command {} \;     (simple, completely robust and portable, may be very slow)
  • find . -exec command {} +       (optimizes speed, quite portable)
  • find . -print0|xargs -0 command      (optimizes speed, if find doesn't implement "-exec +" but knows "-print0")
  • find . -print|xargs command     (if there's no white space in the arguments)
  
  Note: find descends into directories. To avoid that portably, you can use
  • "find . ! -name . -prune [...]"

• If the major part of the arguments consists of long, absolute or relative paths, then try start your actions from within those directories:
  • cd /directory/with/long/path && command *

• And another quick fix may be to match fewer arguments:
  • command [a-e]*; command [f-m]*; ...

Number of arguments and maximum length of one argument

  PAGE_SIZE*MAX_ARG_PAGES-sizeof(void *) / sizeof(void *)

Since Linux 2.6.23, this function tests if the number exceeds MAX_ARG_STRINGS in <linux/binfmts.h> (2^32-1 = 4294967296-1).

And as additional limit since 2.6.23, one argument must not be longer than MAX_ARG_STRLEN (131072).
This might become relevant if you generate a long call like "sh -c 'automatically generated with many arguments'".
(pointed out by Xan Lopez and Ralf Wildenhues)

Actual values for ARG_MAX (or NCARGS)

The maximum length of arguments for a new process is varying so much among unix flavours, that I had a look at some systems:

   # scodb -w
   scodb> maxexecargs=1000000
   scodb> q

   WARNING: table_grow - exec data table page limit of 256 pages (MAXEXECARGS) exceeded by 1 pages
   WARNING: Reached MAXEXECARGS limit while adding arguments for executable "ls"

    "Increase ARG_MAX so that `make clean' in src/lib/libc works again.
    (Adding YP pushed it over the limit.)"

  "((sizeof(struct arg_hunk *))*(0x10000/(sizeof)(struct arg_hunk)))
   space for other stuff on initial stack like aux vectors, saved
   registers, etc.."  

  "ARG_MAX is calculated as follows:
   NCARGS - space for other stuff on initial stack
   like aux vectors, saved registers, etc.."

More pages

• The GNU Core Utilities FAQ from Bob Proulx
• Greg Wooledge's page in his bash faq
• Era Eriksson's pages

Thanks to Gunnar Ritter for the test results from UnixWare and OpenUnix.
Thanks to Rodolfo Martin for access to OpenServer 6.

Comments please to mascheck@in-ulm.de
<http://www.in-ulm.de/~mascheck/various/argmax/>