Problem
This is what I see at the beginning of Python files:
Without that line, the files appear to run the same.
Asked by john garcias
Solution #1
If you have multiple versions of Python installed, /usr/bin/env will use the first interpreter on your environment’s $PATH. Hardcoding something like #!/usr/bin/python is another option, but it’s less versatile.
In Unix, an executable file that’s meant to be interpreted can indicate what interpreter to use by having a #! at the start of the first line, followed by the interpreter (and any flags it may need).
Of course, if you’re talking about other platforms, this restriction doesn’t apply (but that “shebang line” does no harm, and will help if you ever copy that script to a platform with a Unix base, such as Linux, Mac, etc).
Answered by Alex Martelli
Solution #2
The shebang line is what it’s called. According to the Wikipedia entry:
Also see the entry in the Unix FAQ.
You can give options to the interpreter by providing them on the shebang line even on Windows, where the shebang line does not select the interpreter to be run. In one-off scripts (like the ones I make when answering queries on SO), I keep a generic shebang line so I can rapidly test them on both Windows and ArchLinux.
You can use the env utility to run a command on the path:
Answered by Sinan Ünür
Solution #3
Expanding on the other replies, here’s an example of how careless use of /usr/bin/env shebang lines can lead your command line scripts into trouble:
$ /usr/local/bin/python -V
Python 2.6.4
$ /usr/bin/python -V
Python 2.5.1
$ cat my_script.py
#!/usr/bin/env python
import json
print "hello, json"
$ PATH=/usr/local/bin:/usr/bin
$ ./my_script.py
hello, json
$ PATH=/usr/bin:/usr/local/bin
$ ./my_script.py
Traceback (most recent call last):
File "./my_script.py", line 2, in <module>
import json
ImportError: No module named json
The json module doesn’t exist in Python 2.5.
Using the versioned python command names that are normally installed with most Pythons is one approach to avoid such a problem:
$ cat my_script.py
#!/usr/bin/env python2.6
import json
print "hello, json"
If you just need to differentiate between Python 2.x and Python 3.x, new Python 3 releases have a python3 name:
$ cat my_script.py
#!/usr/bin/env python3
import json
print("hello, json")
Answered by Ned Deily
Solution #4
We must give the shell three things in order for the python script to run:
The whole shebang #! gets the job done (1.). The shebang begins with a # because the # character is a comment marker in many scripting languages. The contents of the shebang line are therefore automatically ignored by the interpreter.
The env command satisfies (2) and (3). (3.). “Grawity,” to use a phrase,
Answered by Rose Perrone
Solution #5
The exec system call of the Linux kernel understands shebangs (#!) natively
When you’re on bash, you should:
./something
This invokes the exec system call with the path./something on Linux.
On the file provided to exec, this line of the kernel is called: https://github.com/torvalds/linux/blob/v4.8/fs/binfmt script.c#L25
if ((bprm->buf[0] != '#') || (bprm->buf[1] != '!'))
It reads the file’s very first bytes and compares them to #!
If the comparison is correct, the Linux kernel parses the rest of the line and issues another exec call with:
therefore equivalent to:
/usr/bin/env python /path/to/script.py
env is an executable that scans PATH for /usr/bin/python, for example, and then calls:
/usr/bin/python /path/to/script.py
The #! line in the file is recognized by the Python interpreter, but # is the comment character in Python, thus that line is simply discarded as a regular remark.
Yes, it is possible to create an infinite loop with:
printf '#!/a\n' | sudo tee /a
sudo chmod +x /a
/a
Bash notices the mistake:
-bash: /a: /a: bad interpreter: Too many levels of symbolic links
It’s only that #! is human readable, but it’s not essential.
The exec system call would use a different handler if the file started with different bytes. For ELF executable files, the other most essential built-in handler is: https://github.com/torvalds/linux/blob/v4.8/fs/binfmt elf.c#L1305 which checks for bytes 7f 45 4c 46 (which also happens to be human readable for .ELF). Let’s confirm that by reading the 4 first bytes of /bin/ls, which is an ELF executable:
head -c 4 "$(which ls)" | hd
output:
00000000 7f 45 4c 46 |.ELF|
00000004
So when the kernel sees those bytes, it takes the ELF file, puts it into memory correctly, and starts a new process with it. Also see: How does kernel get a linux executable binary file to run?
Finally, the binfmt misc method allows you to add your own shebang handlers. You can add a custom handler for.jar files, for example. Handlers by file extension are also supported by this technique. Another use for QEMU is to run executables from different architectures transparently.
I don’t think POSIX specifies shebangs however: https://unix.stackexchange.com/a/346214/32558 , although it does mention in on rationale sections, and in the form “if executable scripts are supported by the system something may happen”. macOS and FreeBSD also seem to implement it however.
PATH search motivation
One of the main reasons for the existence of shebangs is because we frequently want to run commands from PATH on Linux, such as:
basename-of-command
instead of:
/full/path/to/basename-of-command
But then, without the shebang mechanism, how would Linux know how to launch each type of file?
In commands, the extension is hardcoded:
basename-of-command.py
Alternatively, any interpreter might use PATH search:
python basename-of-command
would be an option, but it has the big drawback of causing everything to break if we decide to rework the command into another language in the future.
Shebangs are an excellent solution to this problem.
pyenv and other version managers are two of the most common uses of env.
The use of version managers with pyenv is a good example of why you should use #!/usr/bin/env python instead of just /usr/bin/python.
pyenv makes it simple to install various Python versions on a single machine, allowing you to more quickly recreate other projects without the need for virtualization.
The “current” Python version is then managed by placing it in the PATH: A pyenv controlled python, for example, might be found at: apt-get install for multiple python versions
/home/ciro/.pyenv/shims/python
so nowhere close to /usr/bin/python, which some systems might deal with via update-alternatives symlinks.
Answered by Ciro Santilli 新疆再教育营六四事件法轮功郝海东
Post is based on https://stackoverflow.com/questions/2429511/why-do-people-write-usr-bin-env-python-on-the-first-line-of-a-python-script