This post originally appeared in Human Infrastructure, a weekly newsletter about life in IT. A subscription is included with a Packet Pushers membership available at ignition.packetpushers.net.
In my previous Python forays, I’d worked mostly with Linux. Now, as an Apple user, I thought I’d develop in Python natively on my Mac, MacOS being also Unix-y?
MacOS is bundled with Python2, but Apple doesn’t keep the version especially current. For example, a Mojave 10.14.1 box comes with Python 2.7.10, which dates back to May 23, 2015.
I want to work with a current version of the Python interpreter. And since I am starting on fresh projects, I thought I’d work with Python3 wherever possible.
With visions of network automation and chatbots dancing in my head, I set about updating the Python environment on my MacBook. So began my journey of discovery.
I’ve dabbled in native Python on this particular MacBook in the past. Brew, the community-driven package manager for Mac, was there. PIP, the Python package manager was there. Old, moldy versions…but they were there.
Oh, right. PIP. I thought I could quickly install napalm without worrying about the rest of the Python environment. But PIP wasn’t able to install napalm or any other Python packages I asked for, throwing an error related to certificate security when it attempted to connect to any repository for any package. I couldn’t even get PIP to upgrade itself. Hmm.
Some digging later, I found that the certificate issue was known in PIP 1.5.6 and resolved in later versions. Because PIP 1.5.6 was ancient anyway, I didn’t try to fix it. The right answer was to upgrade PIP. But how, since PIP couldn’t even upgrade itself?
Several searches suggested I uninstall and reinstall PIP, which didn’t solve the problem. Several more searches later, I realized that the latest version of PIP is bundled with the latest version of Python. Fantastic.
I decided to simply upgrade Python, which I’d wanted to do anyway. I tried to do this with brew, which failed for obscure reasons I don’t recall. I realized brew hadn’t been maintained by yours truly, so once again, no problem. I attempted a “brew upgrade” to bring brew up to date, and would try to upgrade Python again.
In CPU years, I lived and died several times while ruby churned away, but eventually, I had several screens full of added, removed, and updated brew recipes. Wonderful. With an upgraded brew installation, I should be able to upgrade Python, which would update PIP. And then I should be able to get napalm and whatever else I wanted.
Brew did install Python3 for me, which put my MacBook in a weird Python situation.
Apple ships MacOS with Python2, as I mentioned earlier. Brew had just installed a parallel Python 3.7.2. Brew also installed a bunch of symlinks pointing to various frameworks all over the file system, some of which I had to hack on with chmod just to get the permissions right so that the brew recipe could complete the Python3 package installation without error. Yikes.
I had run into some of the issues Bradford highlights below.
Yes. I got Python3 installed on my MacBook in the end. And…Python3 solved the PIP problem. Solving the PIP problem allowed me to install napalm. However, I don’t like the dual-Python symlink-hell environment I’ve ended up with, RE: XKCD 1987.
The Twitterverse had many comments for me as my Python/Mac journey is hardly a new one. People offered different ways to create a less fragile Python development environment.
1. VirtualEnvWrapper. https://virtualenvwrapper.readthedocs.io/en/latest/. This tool creates a virtual environment that is walled off from your core system or other virtual environments. You can install what you like in here without blowing up anything on your main system. In my case, I could have installed Python3 and whatever modules I wanted without creating XKCD 1987. The learning curve did not look steep to me.
2. Pipenv. https://github.com/pypa/pipenv/blob/master/README.md. Quoting the readme, “Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world…It automatically creates and manages a virtualenv for your projects, as well as adds/removes packages from your Pipfile as you install/uninstall packages.”
Sounds promising, although my take from a couple of tweets (not a scientific survey) is that the Python community has mixed opinions about pipenv.
3. Docker. https://www.docker.com/products/docker-engine. Container fans pointed out that using the Docker Engine, which comes with Docker Desktop for Mac, I could build containerized versions of an environment to develop in Python. There’s more of a learning curve here, but that would have its advantages as the container form factor is here to stay, although I have my doubts about Docker itself.
4. Vagrant. https://www.vagrantup.com/intro/index.html. My understanding of Vagrant is mostly, “In a demo, I saw someone type vagrant up, and then the stars exploded gloriously while magic happened.” In other words, I haven’t invested the time to figure Vagrant out yet.
But I get why Vagrant might apply to me, based on the introductory documentation. “Vagrant provides easy to configure, reproducible, and portable work environments. If you are a developer, Vagrant will isolate dependencies and their configuration within a single disposable, consistent environment, without sacrificing any of the tools you are used to working with (editors, browsers, debuggers, etc.). Once you or someone else creates a single Vagrantfile, you just need to vagrant up and everything is installed and configured for you to work.”
5. PyCharm. https://www.jetbrains.com/pycharm/. PyCharm is an integrated development environment (IDE) for Python, and makes it easier to manage Python interpreters. I don’t think PyCharm ships with a Python interpreter (I could be wrong), but managing whatever interpreters you do have installed looks simple and flexible.
6. Local virtual machine. Whether VirtualBox or VMware Fusion, I have the ability to create a virtual machine and run it on my Mac. No problem. The networking is sometimes painful depending on what I’m trying to do, but for the most part VMs run adequately on a workstation.
7. Remote virtual machine. I can stand up a small Linux box in the public cloud easily and cheaply. Off the top of my head, I could do any of these for $5 a month:
- AWS Lightsail instance with 1GB memory, 1 core, 40GB SSD, and 2TB transfer.
- Vultr Cloud Compute VC2 instance with 1GB memory, 1 core, 25GB SSD, and 1TB transfer
- DigitalOcean standard droplet with 1GB memory, 1 core, 25GB SSD, and 1TB transfer
Other charges will pop up for exceeding storage and transfer limits, but in principle I could have a box to hack around with for $5 a month. There are $3.50 a month options if you look.
Vultr even offers a $2.50 a month IPv6-only VC2 instance option, which would be spectacular if my stuck-in-the-past regional ISP offered v6. (I know, I know…tunnelbroker.net to work around v6 poverty. Sigh.)
The remote VM appeals to me. I can access the instance anywhere I have Internet. I can give other people access. I want to work with chatbots, and a remote virtual machine makes it easier to keep the chatbot attached as compared to a laptop or workstation.
Of course, using a remote virtual machine does not exclude VirtualEnvWrapper, Docker, or Vagrant.
We’ll see where I go next as I explore Python development in a way that makes sense for how I work and what I’m trying to do.