Packaging Python Apps For Windows

Building a packaged python executable in Windows with py2exe

After spending considerable time on getting a working Windows setup for Pronterface a random person on IRC convinced me to document it for others who might have the same problem. This is a guide to creating a self-contained windows executable from a python script. This is useful for distributing a script to users who do not have python and/or all necessary libraries installed. An executable made this way also enables 32-bit only Python libraries to run correctly on 64-bit Windows installations if built on a 32-bit Windows machine. This guide assumes you already have a python install and a functional program you intend to distribute. The user will not need administrator priviledges on their machine to use the resulting executable.

Prepare your program

Make sure your program runs correctly. It is much more difficult to debug an already exe-ified program.

Install py2exe.

You can get py2exe from the py2exe sourceforge page

Create a setup.py file for your project.

This file describes the structure of your project to py2exe so it knows which files to include. Put it in the same directory as your script. Here is an example setup.py:

from distutils.core import setup
import py2exe

setup(
    windows=[{'script':'filename.py'}],
    options={'py2exe':{'bundle_files':1,'compressed':1}}
)

Let’s go through it line by line:

from distutils.core import setup
import py2exe

This imports the basic setup function from distutils, which checks the dependencies of your script and determines which other modules mush be included for your script to function.

setup(
    windows=[{'script':'filename.py'}],
    options={'py2exe':{'bundle_files':1,'compressed':1}}
)

Here, we call the setup function and give it a number of options. The “windows” parameter defines which script file corresponds to a windowed application. In this case, it’s set to “filename.py”. Substitute the name of the script you are building. Windows makes a distinction between console and windowed applications. If your application needs a console (even if it does GUI stuff as well) you might want to define it as a console application rather than a windowed one. To do this, replace windows with console:

setup(
    console=[{'script':'filename.py'}],
    options={'py2exe':{'bundle_files':1,'compressed':1}}
)

You can also have more than one script, and thus generate more than one executable:

setup(
    windows=[{'script':'filename1.py'},{'script':'filename2.py'}],
    options={'py2exe':{'bundle_files':1,'compressed':1}}
)

You can also combine any number of windowed and console applications:

setup(
    windows=[{'script':'w1.py'},{'script':'w2.py'}],
    console=[{'script':'c1.py'},{'script':'c2.py'}],
    options={'py2exe':{'bundle_files':1,'compressed':1}}
)

You might have noticed the entries in windows and console are dictionaries. You can add extra options in there that affect each individual file. One particularly useful one is adding an icon to the executable.

setup(
    windows=[{'script':'filename.py','icon_resources':[(1,'iconname.ico')] } ],
    options={'py2exe':{'bundle_files':1,'compressed':1}}
)

You can add any number of icons, but only the first one will be displayed by default. The “options” parameter allows us to pass options directly to py2exe. Let’s look at the options there in more detail:

    options={'py2exe':{'bundle_files':1,'compressed':1}}

bundle_files tells py2exe to collect all the files that the script uses together into a bundle. That way you have to distribute fewer files. All the python libraries and modules that your script uses will be collected into a single file called library.zip by default. Compressed tells py2exe to compress the file so it takes up less space. This is nice when you have downloadable files. When you run the executable, it will automagically access the files it needs from within the compressed bundle without having to unpack them anywhere. You can specify the name of the file where all the libraries and modules are stored, like this:

setup(
    windows=[{'script':'filename.py'}],
    zipfile='blob.zip',
    options={'py2exe':{'bundle_files':1,'compressed':1}}
)

Now instead of library.zip all the libraries and modules will be stored in blob.zip. If you only have one executable, you can do one better. You can include all the libraries and modules directly in the executable itself. You can do this with multiple executables also, but then each one will have its own copy of all the libraries, and will be much larger as a result. To do this, use:

setup(
    windows=[{'script':'filename.py'}],
    zipfile=None,
    options={'py2exe':{'bundle_files':1,'compressed':1}}
)

This way, you will only have a single executable with no other files.

Add supporting files

Suppose you have some supporting files you want to include with your script. These can include images, libraries python does not know about, sounds, other content files, or anything else that is not part of the code your program runs. You can tell py2exe to copy those over to the same directory where your exe gets generated. Unfortunately, additional files cannot be easily included in the bundle file, because they must be unpacked somewhere before the script can access them. Here is how you copy them over:

dfiles = [(".",["f1.dat"]),("logs",[]),('img', ["i1.png", "i2.png"]), ("snd",["s1.wav","s2.wav"])]
    
setup(
    windows=[{'script':'filename.py'}],
    data_files=dfiles,
    options={'py2exe':{'bundle_files':1,'compressed':1}}
)

This will copy the file “f1.dat” to the base directory of the executable, create an empty folder called “logs”, create a folder called “img” and copy i1.png and i2.png into it, and then create a folder called “snd” and copy s1.wav and s2.wav into it.

Create a batch file for convenience

In the folder where your setup.py script is, create a text file called setup.bat with the following content:

C:\\python27\python setup.py py2exe -v
pause

Substitute the path to your Python version. When you double-click this file, it will run py2exe and show you the output. The -v makes py2exe spit out extra information about what it’s doing. The pause command makes sure that Windows does not hide the output window as soon as it’s done, giving you time to read the output.

Build your executable

Double-click the setup.bat file. It will run py2exe, which will then create a build directory. The files needed by your script will be copied into that directory. Once they are all there, they will then be bundled and the bundle and executable files, as well as all the supporting files you specified, will be copied into a directory called “dist”. The contents of this directory are what you have to distribute.

Add necessary DLL files

When you run the setup.bat file, it will show you a list of DLL files needed for your program to run. If any of those include a file that starts with msvcp or msvcr then the users of your program will need to have the same one installed. It is a good idea to copy these files into your dist directory before you distribute it. Most of the other files are included with Windows by default and do not need to be copied.

Test your program

Go into the dist directory where your executable now lives, and try to run it. See whether everything works correctly. Copy the dist directory to the target machine and test it there.

Additional resources

The py2exe page has a lot of additional information about more advanced use cases.

Posted in  software


4pi Firmware

I’ve been spending some time in the last weeks working on porting Sprinter over to my new SAM3U-based 4pi electronics. I’ve extracted several useful elements into libraries (sort of) so they can be used for other projects too.

ADC - samadc.c

The 12 bit ADC built into the SAM3U has 8 channels. 4 of them (channels 5,3,1, and 2) are wired to the thermistor voltage dividers. Channels 4, 6 and 7 are connected to the X, Y, and Z max endstop pins, but can be used as ADC channels as well. They are disabled by default. Channel 0 is not available on 4pi.

To use the ADC, its initialization function is called with an argument that specifies whether the ADC is to take continuous readings or only make a measurement when one is requested:

void initadc(int autos);

Where autos is 0 for manual sampling and >0 for continuous sampling.

unsigned int adc_read(unsigned char channel);

Where channel is a number from 1 to 7. The value returned is in millivolts. adc_read will never block. It will always return the last value sampled on any of the enabled channels. If you are running in continuous mode, you can rely on this being a very recent value. If not running in continuous mode, you have to call a function to initiate a read:

void adc_sample();

This will make a measurement on all the enabled channels. In Sprinter, I’ve set it up to make a measurement 100 times per second.

Motor options - motoropts.c

The 5 motor drivers on the 4pi all have adjustable current levels and microstepping settings. This file abstracts these settings and lets you control the stepper drivers.

Before you use the motor drivers, they must be initialized. Use the

void motor_setup();

function to do this. This sets each motor to 1/16th microstepping, at half current. All motors are disabled on startup.

To adjust the microstepping or current settings of an axis, use

void motor_setopts(unsigned char axis, unsigned char ustepbits, unsigned char current);

axis is a number from 0 to 4, corresponding respectively to X, Y, Z, E0 and E0 drivers. The AD5206 digital potentiometer has six channels, and the three pins of the remaining channel are brought out to solder pads on the board. You can use axis number 6 to adjust its current. ustepbits are the microstepping settings on the stepper drivers. The values are 0 for full step, 1 for half step, 2 for 1/4 step and 3 for 1/16 step. You can put in 4 here to leave the microstepping unchanged and adjust only current. current is an integer from 0 to 255 that adjusts the current limit of the motor. 255 is full power, 0 is off.

There is a function to enable or disable an axis as well:

void motor_enaxis(unsigned char axis, unsigned char en);

Where axis is as above and en is 0 for disable and >0 for enable. A similar function exists for setting the direction:

void motor_setdir(unsigned char axis, unsigned char dir);

Finally, to step an axis, use:

void motor_step(unsigned char axis);
void motor_unstep();

motor_step will set the step pin for a given axis high, motor_unstep will set all axes low. These functions will always be inlined.

Heater/FET control - heaters.c

The heater and FET control is fairly simple. You initialize the FETs with the

heaters_setup();

function. All heaters are off on startup. After that, you can switch each channel with the

heater_switch(unsigned char heater, unsigned char en);

function. If en==0, the FET will be switched off. If en>0, it will be switched on. The FET IDs are:

0 - Bed heater
1 - Extruder 0 (E0)
2 - Extruder 1 (E1)
3 - AUX0
4 - AUX1

Systick timer

The SAM3U has an ARM Cortex-M3 processor. That processor has a systick timer that counts down from a given 24bit value to zero on either every clock tick or every eighth clock tick of the master clock. It can generate an interrupt whenever the counter reaches zero, and gets automatically reset to the initial value. This is a nice way to get a periodic timer “for free” as it leaves the other three timers on the chip free for more sophisticated stuff. I use the systick timer to generate an interrupt every millisecond. This currently drives the temperature measurement as well as updating a clock that a number of functions in Sprinter use.

USB-Serial - serial.c

The SAM3U has a native high speed USB controller running at 480MHz. This required some creative routing to attach to the USB port as traces carrying signals at that speed need to be precisely length-matched:

Length-matched traces

I implemented a virtual serial port running over the USB link, that shows up as an ACM device. Most of the work is done by libs Atmel helpfully provides (with a BSD license too!) in their at91lib. Here are the functions I wrote:

void samserial_init();
void samserial_print(const char* c);
void samserial_setcallback(void (*c)(unsigned char));

samserial_init initializes the serial port with a default baudrate of 115200bps (I’ll make that configurable later). samserial_print attempts to write a string to the virtual serial port. samserial_setcallback takes a function pointer and calls the function whenever a new character is received on the serial port. The Sprinter flow control code and checksum calculator gets attached to this callback.

Posted in  electronics


Peristaltic Pump

This is the first component of a what I hope to become a completely unattended yarn dyeing system.

The problem

Yarn is available in many materials and colors, but somehow it’s still troublesome to find something that’s just right for a project. Hand-dyed yarn gives beautiful and finely controllable results, but is not easily repeatable or controllable. I want to build a machine that takes a cone of yarn and produces dyed, washed, and dried balls of beautiful yarn, repeatable indefinitely.

Process

The yarn path defines the gross structure of the machine:

  • The machine pulls yarn off the cone and feeds it into a dye bath. The heated dye and fixative mix is absorbed into the yarn.
  • (possibly the yarn should be heated additionally at this point)
  • The yarn continues into a water bath, where it is rinsed from unfixed dye.
  • The yarn passes through multiple jets of hot air, which dry it.
  • The yarn is then pulled onto a ball winder or cone winder.

In addition, a color mixing system is needed, that can continuously feed precisely controlled volumes of each dye. The passing yarn slowly depletes the dye pool, allowing newly added dye to create smooth color transitions. The dye must be either heated before being deposited in the dye pool. or heated inside the dye pool itself.

The pump

The central part of the dye mixer is a set of pumps. These need to fulfill several requirements: -They must be positive displacement pumps. That is, the pump needs to move known quantities of dye rather than just creating a pressure differential. -They must be easy to clean. Dye is messy. Contamination is unpleasant. -They must be acid-resistant. The most useful dye type has rather low pH. -They must be liquid-proof.

Peristaltic pumps, used in the medical industry and food processing, would fulfill all of these requirements. That’s what I decided to try first. A peristaltic pump has a number of rollers that make contact with the outside of a tube, compressing said tube against a wall. As the rollers move, the compressed section of tubing is pushed along the tube, sealing against itself and transporting a fluid forward. The major disadvantage of this is that the output pulses, having lower pressure during the open sections and higher pressure during the compressed sections. To reduce the effects of said pulsing, I’m using five separate rollers to make it pulse at five times the frequency.

Design iteration 1

The pump consists of a rotor and a body. The rotor itself has a top and bottom part, and 624ZZ bearings fit between the top and bottom and are held in with M4 bolts. The tubing is held between the motor body and the bearings on the rotor. The rotor has a nut trap for an M3 nut. An M3x10 bolt attaches the rotor to the 5mm motor shaft. The body is built around a NEMA14 motor. It has 4 mounting holes for the motor bolts and a hole for the motor shaft, as well as mounting holes for attachment to structural components of the machine.

Problems:

-The tubing is not compressed sufficiently by the bearings - increase the distance from bearing to rotor. -The tubing gets pulled tight between the bearings - increase the number of bearings from 3 to 5 to reduce the chord length. -The NEMA14 motor does not have enough torque - even a NEMA17 barely does. Tricky. A gear would be in order here.

Posted in  designs