HomeI am developing PyTiCroque which is python + Qt based software. The main platforms are Linux and Windows.
When I am close to final release, getting the release in shape usually goes through several attempts: I generate one release file, test it, find a bug, fix it, generate another release file and here we go again.
Of course, I have automated most of the process. I have a small python script make_exe.py that take care of most of the it: exe building on windows with PyInstaller, then installer building with InnoSetup.
There is one step that I had not automated so far: uploading the release file to RedMine. I used to go through manual upload but for the last release, I had to release the file several times and was annoyed by this last manual step. Surely, there is something I can do to automate this.
It’s been a long time that I wanted to touch web scraping and web automation. After a quick search, I found mechanize and it did the job very well. The doc are a bit strange, with no autogenerated docs, and just example on the website, but the tool is so simple to use that it was not much of a problem.
In 15 minutes, I had my script running:
- Login to Redmine
- Go to the File Upload page
- Enter the information for the new file: filename, mimetype, release attached to the file
- upload it
The final script is:
url_fh = 'http://labs.freehackers.org/'
br = mechanize.Browser()
br.set_handle_robots(False)
report( 'Fetching labs.freehackers.org/login page...' )
page = br.open( url_fh + 'login' )
report( 'Logging in...' )
br.select_form( nr=1 )
br["username"]="philippe"
br["password"]="XXXXXX"
br["autologin"]=0
page = br.submit()
report( 'Opening PyTiCroque - New Files page...' )
page = br.open( url_fh + 'projects/pyticroque/files/new' )
report( 'Registering the file' )
br.select_form( nr=1 )
br["version_id"]= [ "44" ]
br.add_file( open( r'd:\work\pyticroque-dev\pyticroque\LISEZMOI.txt' ), "text/plain", "LISEZMOI.txt" )
page = br.submit()
report( 'File sumitted !' )
The process is so simple that I plan to automate more stuff in the future. For example, my users are so not computer savy that doing through a bug reporting software is too much for them. Still, I like good bug reports. I have a log file that gives me accurate information about everything that happened in the software, so if they can attach that file to the bug report, I can probably find the bug. But attaching a file to a bug report is not so simple for non computer savvy people. Instead, what I plan to do is automate the process of bug reporting with a wizard. They will go through the following steps:
- You want to report a bug ?
- You want to be notified of updates ? If so, please enter your email address ?
- Click [Send bug report]
Behind the scene, I will connect to Redmine, identify the right version, prepare the bug report form, attach the log file and enter it in the bug manager. I hope I can get the bug reports then !
April 9th, 2010 at 18:15
[...] premier script en python… mis au point grâce à l’aide de ce billet. On y apprend comment installer les librairies mechanize et ClientForm, et comment les utiliser [...]
July 23rd, 2011 at 16:23
[...] [...]
November 4th, 2011 at 19:44
Hello,
please see the Perl version below, just for fun. Server project filename from ARGV, user passwd from stdin.
#!/usr/bin/perl
use strict;
use WWW::Mechanize;
use Term::ReadKey;
print “username: “; (my $username = ) =~ s/[\r\n]//g;
ReadMode ‘noecho’;
print “password: “; (my $password = ) =~ s/[\r\n]//g;
ReadMode 0;
my ($server, $pj, $file) = @ARGV;
my $mech = WWW::Mechanize->new();
$mech->get(“http://$server/redmine/login”);
$mech->submit_form(with_fields => { username => $username, password => $password });
$mech->get(“http://$server/redmine/projects/$pj/files/new”);
$mech->submit_form(with_fields => { “attachments[1][file]” => $file });
January 18th, 2012 at 22:20
Hi,
I’ve used your code sample to upload files to my Redmine instance. I’m running into a problem though: mechanize is failing to POST the request appropriately. The message should be a multipart/form-data, but when I upload a file only the first 79 bytes are sent. Have you run over any issues like this?
January 23rd, 2012 at 19:14
Silly me, forgot to open the file in binary mode and line buffering caused the file.read method to return just the first 79 bytes of the archive.