Getting Started

In this section you will:

  • Sign up for Github.

  • Generate a scaffold for your new Python project.

  • Upload it to GitHub.

  • Install your new Python project for development.

Then, in the next section, we will begin to move your scientific code into that template.

You will need a Terminal (Windows calls it a “Command Prompt”) and a plain text editor. Any will do; we won’t assume anything about the editor you are using. If you are looking for a recommendation for beginners, the Atom Editor by GitHub is a good one. For minimalists, nano on Linux or OSX and Notebook on Windows will get you up and running.

Reading the steps that follow, you might reasonably wonder, “Why isn’t there just an automated script for this?” We prefer to do this process manually so that we are forced to think carefully about each step, notice when something goes wrong, and debug if necessary. We recommend you do the same.

  1. Sign up for GitHub.

  2. Verify that you have Python 3.

    python3 --version
    

    If necessary, install it by your method of choice: apt, Homebrew, conda, etc.

  3. Create an environment, a sandboxed area for installing software that is separate from the system defaults. This is not essential, but it is strongly encouraged. It ensures that your project and its software dependencies will not interfere with other Python software on your system. There are several tools for this. But the simplest is Python’s built-in venv (short for “virtual environments”), illustrated here.

    Do this once:

    python3 -m venv my-env
    

    The term my-env can be anything. It names the new environment.

    Do this every time you open up a new Terminal / Command Prompt to work on your project:

    . my-env/bin/activate
    

    Note

    If you are a conda user, you may prefer a conda environment:

    conda create -n my-env python=3.6
    conda activate my-env
    
  4. Verify that you have git installed.

    git
    

    If necessary, install it by your method of choice (apt, Homebrew, conda, etc.).

  5. Choose a name for your project.

    Ideal names are descriptive, succinct, and easy to Google. Think about who else might be interested in using or contributing to your project in the future, and choose a name that will help that person discover your project. There is no need to put “py” in the name; usually your user will already know that this is a Python project.

    Check that the name is not already taken by searching for it on the Python Package Index (PyPI).

  6. Install cookiecutter.

    python3 -m pip install --upgrade cookiecutter
    
  7. Generate a new Python project using our cookiecutter template.

    cookiecutter https://github.com/NSLS-II/scientific-python-cookiecutter
    

    You will see the following the prompts. The default suggestion is given in square brackets.

    For the last question, minimum_supported_python_version, we recommend supporting only back to Python 3.6 so you can use features not available in Python 3.4 or 3.5. But we allow for the possibility that you may need to support older versions depending on your circumstances. Again, you can use python3 --verison to check what version of Python is currently available on your machine.

    full_name [Name or Organization]: Brookhaven National Lab
    email []: dallan@bnl.gov
    github_username []: danielballan
    project_name [Your Project Name]: Example
    package_dist_name [example]:
    package_dir_name [example]:
    repo_name [example]:
    project_short_description [Python package for doing science.]: Example package for docs.
    year [2018]:
    Select minimum_supported_python_version:
    1 - Python 3.6
    2 - Python 3.5
    3 - Python 3.4
    Choose from 1, 2, 3 [1]:
    

    This generates a new directory, example in this case, with all the “scaffolding” of a working Python project.

    $ ls example/
    AUTHORS.rst        MANIFEST.in     example                 setup.cfg
    CONTRIBUTING.rst   README.rst      requirements-dev.txt    setup.py
    LICENSE            docs            requirements.txt        versioneer.py
    

    Note

    Cookiecutter prompted us for several variations of name. If are you wondering what differentiates all these names, here’s a primer:

    • project_name – Human-friendly title. Case sensitive. Spaces allowed.

    • package_dist_name – The name to use when you pip install ___. Dashes and underscores are allowed. Dashes are conventional. Case insensitive.

    • package_dir_name — The name to use when you import ___ in Python. Underscores are the only punctuation allowed. Conventionally lowercase.

    • repo_name — The name of the GitHub repository.

  8. Take a moment to see what we have.

    example/
    ├── .flake8
    ├── .gitattributes
    ├── .gitignore
    ├── .travis.yml
    ├── AUTHORS.rst
    ├── CONTRIBUTING.rst
    ├── LICENSE
    ├── MANIFEST.in
    ├── README.rst
    ├── docs
    │   ├── Makefile
    │   ├── build
    │   ├── make.bat
    │   └── source
    │       ├── _static
    │       │   └── .placeholder
    │       ├── _templates
    │       ├── conf.py
    │       ├── index.rst
    │       ├── installation.rst
    │       ├── release-history.rst
    │       └── usage.rst
    ├── example
    │   ├── __init__.py
    │   ├── _version.py
    │   └── tests
    │       └── test_examples.py
    ├── requirements-dev.txt
    ├── requirements.txt
    ├── setup.cfg
    ├── setup.py
    └── versioneer.py
    

    In this top example/ directory, we have files specifying metadata about the Python package (e.g. LICENSE) and configuration files related to tools we will cover in later sections. We are mostly concerned with the example/example/ subdirectory, which is the Python package itself. This is where we’ll put the scientific code. But first, we should version-control our project using git.

  9. Change directories into your new project.

    cd example
    

    We are now in the top-level example/ directory—not example/example!

  10. Make the directory a git repository.

    $ git init
    Initialized empty Git repository in (...)
    
  11. Make the first “commit”. If we break anything in later steps, we can always roll back to this clean initial state.

    $ git add .
    $ git commit -m "Initial commit."
    
  12. Create a new repository on GitHub, naming it with the repo_name from your cookiecutter input above.

    Important

    Do not check “Initialize this repository with a README”.

  13. Configure your local repository to know about the remote repository on GitHub…

    $ git remote add origin https://github.com/YOUR_GITHUB_USER_NAME/YOUR_REPOSITORY_NAME.
    

    … and upload the code.

    $ git push -u origin master
    Counting objects: 42, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (40/40), done.
    Writing objects: 100% (42/42), 29.63 KiB | 0 bytes/s, done.
    Total 42 (delta 4), reused 0 (delta 0)
    remote: Resolving deltas: 100% (4/4), done.
    To github.com:YOUR_GITHUB_USER_NAME/YOUR_REPO_NAME.git
     * [new branch]      master -> master
       Branch master set up to track remote branch master from origin.
    

    Note

    If this repository is to belong to a GitHub organization (e.g. http://github.com/NSLS-II) as opposed to a personal user account (e.g. http://github.com/danielballan) it is conventional to name the organization remote upstream instead of origin.

    $ git remote add upstream https://github.com/ORGANIZATION_NAME/YOUR_REPOSITORY_NAME.
    $ git push -u upstream master
    Counting objects: 42, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (40/40), done.
    Writing objects: 100% (42/42), 29.63 KiB | 0 bytes/s, done.
    Total 42 (delta 4), reused 0 (delta 0)
    remote: Resolving deltas: 100% (4/4), done.
    To github.com:ORGANIZATION_NAME/YOUR_REPO_NAME.git
     * [new branch]      master -> master
       Branch master set up to track remote branch master from upstream.
    

    and, separately, add your personal fork as origin.

    $ git remote add origin https://github.com/YOUR_GITHUB_USER_NAME/YOUR_REPOSITORY_NAME.
    
  14. Now let’s install your project for development.

    python3 -m pip install -e .
    

    Note

    The -e stands for “editable”. It uses simlinks to link to the actual files in your repository (rather than copying them, which is what plain pip install . would do) so that you do not need to re-install the package for an edit to take effect.

    This is similar to the behavior of python setup.py develop. If you have seen that before, we recommend always using pip install -e . instead because it avoids certain pitfalls.

  15. Finally, verify that we can import it.

    python3
    
    >>> import your_package_name
    
  16. Looking ahead, we’ll also need the “development requirements” for our package. These are third-party Python packages that aren’t necessary to use our package, but are necessary to develop it (run tests, build the documentation). The cookiecutter template has listed some defaults in requirements-dev.txt. Install them now.

python3 -m pip install --upgrade -r requirements-dev.txt

Now we have a working but empty Python project. In the next section, we’ll start moving your scientific code into the project.