Here's a simple conundrum: Given a file, say:
/foo/bar/baz.exe
what are the names for the different parts of it?
The directory (part) is /foo/bar. The non-directory or file part is baz.exe. The extension is .exe.
So what is baz?
Frequently, I've written code that produces …
You have a Python script or package that is running perfectly. After you make some alterations, "all of a sudden" you can't run it because it can't find a variable or name:
AttributeError: 'module' object has no attribute 'foobar'
But if you search for that name, it doesn't …
“Suddenly” (as is traditional to commence these sort of stories), I started having problems with some of the Python installations on my local machine. As is the way of things, I had three distinct sources of Python:
IPython notebooks are an incredibly useful way to do reproducible research and illustrate chunks of code. Obviously you'd like to put some of them on the web. Surely there must be an easy way to include them in a Pelican-based website. So why hasn't anyone done it yet?
They have …
You need to convert a time difference into a floating point quantity, like 2.5 days. Or you may want to do something related like round a fdatetime difference up or down, e.g. 3 or 2 days for our previous example.
When you subtract a …
One of the big sells of Pelican is being able to use multiple markdup formats, being able to drop in notebooks and documents written in other contexts. One such format is RMarkdown (.rmd) documents, such as are used in knitr for reproducible analyses, allowing you to do an analysis and …
I try to avoid PHP as much as possible (just not my thing) but I'd installed a PHP-based webapp (REDCap) and there was something occuring that seemed like it should have generated an error. However I could not find any PHP errors logged anywhere on the machine. So …
Way, way back, this site ran on various permutations of PHP-based frameworks. Getting sick of ugly URLs and having to vigilantly update the software so as to avoid hacks (and still periodically falling victim to various hacks), I went in search of a Python-based framework that I could customize …
I'll stick with LaTeX, thank you very much. Which is poking yourself in the eye with a tool perfectly designed for eye-poking after you get the configuration right, but the tool can't be held by human hands so you also need a pneumatic dingus to poke the poker and a …
The first thing to know is that nearly every object in R is really just a list with named elements. For example, the results returned from "summary" are returned in an object:
x <- summary(c(1:10)) x ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 1 …
The various types of OOP in R, whatever that is.
Doing object-oriented programming (OOP) in R can be complicated. This is because when you look in R for the usual idioms and features of OOP:
A more recent development in R is Reference classes. These promise, at last, fully blown objects and classes like those of C++ and Java. Naturally, this innovation comes with some downsides:
This is not as straightforward as it seems.
In development, it works fine. Simply declare that the model has an attachment:
class Download < ActiveRecord::Base
...
has_attached_file :attachment
and stuff will work automagically. A directory “system” will be created in your public folder and the attachments will be created in there …
= Top level title == 2nd level / sub-title ---- A horizontal rule is at least 4 dashes ---- Text can be _italic_, *bold* or +monospaced typewriter+. Verbatim / literal text is indented. But look out if it's after a list, as they must be idented to a different extent ---- * Indent your lists and use stars …
Buried in the Python standard library, unicodedata contains most the information needed to interrogate and translate unicode characters. Unfortunately, it's underdocumented. More accurately, the docs are a terse list of what it does, but not why you might want to use it or how you use it. Unfortunately it's also …
See Problem 8: Find the greatest product of five consecutive digits in a 1000-digit number.
This is fairly straightforward. First we define the number, using Python's implicit string continuation:
numstr = '73167176531330624919225119674426574742355349194934' '96983520312774506326239578318016984801869478851843' '85861560789112949495459501737958331952853208805511' '12540698747158523863050715693290963295227443043557' '66896648950445244523161731856403098711121722383113' '62229893423380308135336276614282806444486645238749' '30358907296290491560440772390713810515859307960866' '70172427121883998797908792274921901699720888093776' '65727333001053367881220235421809751254540594752243' '52584907711670556013604839586446706324415722155397' '53697817977846174064955149290862569321978468622482' '83972241375657056057490261407972968652414535100474' '82166370484403199890008895243450658541227588666881' '16427171479924442928230863465674813919123162824586' '17866458359124566529476545682848912883142607690042' '24219022671055626321111109370544217506941658960408' '07198403850962455444362981230987879927244284909188' '84580156166097919133875499200524063689912560717606' '05886116467109405077541002256983155200055935729725' '71636269561882670428252483600823257530420752963450' …
See Project Euler Problem 112: Find the point at which the porportion of bouncy numbers is 99%.
This is a simple problem to state, but has some tricks. First in the conversion of a number into its component digits. This can be neatly done with the [int (x) for x …
See Problem 7: What is the 10001st prime number?
This solution is fairly simple (and largely stolen from Aaron Gallagher):
def iter_primes ():
# handle trivial case
yield 2
val = 1
primesq_pairs = []
while True:
curr = None
while (curr is None):
val += 2
curr = val
for prime, square in primesq_pairs:
if (curr < square …
For better or worse, programming in Ruby means that you will be spending a lot of time in irb. You can customise its behaviour in your .irbrc file, usually found in your home directory. The tweaks in mine were gathered from various places across the 'net:
require "rubygems" require "wirble …
See Problem 20.
Again, this rescues a lot of code from previous problems, but the solution is less than perfect. The Euler guidelines say that every problem should be computable within a minute, but despite much tweaking, this solution takes significantly longer.
For speed, primes should be generated only once …
The problem is:
Print the sum of numbers less than 1000 that are divisble by 5 or 3.
Already, we're exposed to a few nifty features of Groovy - ranges, and the functional methods on arrays like findAll:
// print the sum of numbers less than 1000 that are divisble by 5 …
The sum of the squares of the first ten natural numbers is, 1^2 + 2^2 + ... + 10^2 = 385. The square of the sum of the first ten natural numbers is (1 + 2 + ... + 10)^2 = 55^2 = 3025. Hence the difference between the sum of the squares of the first …
By listing the first six prime numbers: 2, 3, 5, 7, 11, and 13, we can see that the 6th prime is 13. What is the 10001st prime number?
It seems a bit cumbersome, but use a sieve of erasthonenes to generate primes until we have collected the 10001st. Basically …
Exercise 3: finding prime factors
What is the largest prime factor of the number 600851475143?
// What is the largest prime factor of the number 600851475143? target = 600851475143 factors = [] residue = target residue_is_prime = false // NOTE: ``#`` is not a comment character! // While the residue is not prime, keep extracting factors and adding to …
Exercise: 4 palindromic numbers /* A palindromic number reads the same both ways. The largest palindrome made from the product of two 2-digit numbers is 9009 = 91 x 99. Find the largest palindrome made from the product of two 3-digit numbers. */ // The only way to work this out is to try …
Exercise 5: smallest divisible number
What is the smallest positive number that is evenly divisible by all of the numbers from 1 to 20?
/* 2520 is the smallest number that can be divided by each of the numbers from 1 to 10 without any remainder. What is the smallest positive …
Not the first time I've been stung by this, so it's worth recording the problem.
A function or call inexplicably returns not as the expected type but as a tuple containing the said type. After a furious amount of debugging, you determine that everything is acting as it should …
Naming an association in Rails
In which we thwart the magic of Ruby on Rails.
The automagically generated and named rails associations (:has_many, :belongs) are handy, but I ran into problems when trying to give an association a name other than the default based on the class name (e.g …
Find the sum of all even fibonacci numbers that do not exceed four million
Warning, there's a slightly crude appending to an array below (I learn how to properly append in later exercises). findAll gets another working out:
fibos = []
a1 = 0
a2 = 1
while (true) {
curr_term = a1 + a2
if (4000000 …
Django's form generation machinery for the most part is fine and straightforward, but on a recent project I ran into problems with a form that accepted files. The documentation is all there, but changes in how things work and an emphasis on forms for models make things harder to understand …
RMagick is necessary for a lot of Ruby packages, so the restrictions on using it are critical (and very annoying). Here are the constraints on what can be used where:
| Ruby 1.8 | RMagick 1 | RMagick 1 |
| Ruby 1.9 | RMagick 2 |
Or to put it another way: if you …
Maps are great quick-and-dirty data structures or caches in many programming languages. But there's a least one way that Groovy maps aren't so friendly. Observe:
groovysh> m1 = [1:1]
===> {1=1}
groovysh> m1[1]
===> 1
groovysh> m1[(int) 1]
===> 1
groovysh> m1[(long) 1]
===> null
So, map keys are using …
A common use-case in web development is including some asset (e.g. a stylesheet or javascript file) only for certain pages. The situation gets trickier if the inclusion has to be in the page header. Finding how do do this in RoR is surprisingly tricky, due to outdated or bad …
Setuptools and the single file
Problems with making an installer for a simple module
Although setuptools is a complex thing, and there has has been widespread complaints about its architecture, for the most part, I've had few problems with it. However, I recently encountered a crusty and under-documented corner of …
It's often useful for a script to allow the specification of how an output file is named, e.g. myscript.rb -o file.out. At the same time, it would be nice to have the output name based on the an input file name, and have easy and consistent naming …
How to compile a MacOS executable from C++ using the Netbeans IDE.
Universal binaries are one of Apple's clever innovations that paper over the differences between architectures by including several distinct executables within a single file. When run, the OS examines the file and decides which binary to execute. Thus …
Despite the Plone community's tendency to dump everything and jump onto the latest version, I'd resisted the move to Plone 3.0. One good reason for this is that I'd repeatedly installed Plone 3.0 and it had - repeatedly - failed to startup. The symptoms weren't identical every time, but most …
In which we consider the difference between or and || in Ruby. The secret is ... it depends. On context.
Behold:
>> foo = nil or "foo" => "foo" >> bar = nil || "bar" => "bar"
That makes sense. But!:
def testor (&block) foo = block or "foo" bar = block || "bar" return foo, bar, block end >> testor() => [nil, "bar …
Amongst the many entries Ruby has in the "you have got to be kidding me" stakes, this is a doozy.
Due to Ruby's lack of explicit support for keyword arguments, it's traditional to use a quirk of its argument parsing that pushes named arguments into a hash:
def foo (a …
In a way, tuples are one of the most magical parts of Python. How does the interpreter distinguish between a method call, a bracketed portion of a mathematical expression and an actual tuple?:
calc_midpoint (1, None, [2, 3], False) x = w + (x * y) a = (b, c)
The really smart thing …
In which I discover things, sometimes painfully.
Unless otherwise stated, done using Ruby 1.9.2, Hobo 1.0.3 (and hence Rails 2.3.10):
It seems that permissions in Hobo can be a little tricky, in so much as I've had a case where something that behaved …
Some (very) quick notes on libraries for manipulating (mostly reading) Excel files in Ruby.
Hobo is a nifty thing, a better Rails than Rails, but there are a number of small roadbumps to getting a project started. This is a list of one possible way through the maze.
So I was writing a web application using WSGI and - after working for a long time - it started erroring out with:
AssertionError: write() argument must be string
from deep inside the WSGI handler. Now, this usually means that you're returning unicode, which WSGI can't handle. However, it's quite happy to …
In several recent projects, I've needed to produce graphs of various epidemiological & bioinformatic datasets on demand in web applications. Unfortunately, in the world of Ruby and Rails, the choices of toolkits aren't as straightforward as hoped. This is a summary of the available choices and what I found.
If you need to draw a custom graph type in Ruby, there's a few problems:
Which leaves you few choices. You could manually draw it …
Who would of thunk it? print is a reserved word everywhere.
Perhaps this is buried in some specification, but it seems that you can't use print as the name of a method of a class. (It makes sense you can't override the global name, but in an object? Perhaps it's …
Recently, I had the need to get an informative name from a range of Python objects for generating a useful error message. Where problems started was that these objects could include classes (new-style and old), functions, lambdas and built-in types. And here the logic started getting tricky.
So, epydoc used to be the neatest and bestest documentation tool available for Python. It produced documentation from introspecting code and thus was the easiest way to the most accurate API documentation. However, it looks like the project has been abandoned. Attempts to run it over code result in sometimes …
Let me explain: Ete2, a Python module for representing phylogenies, has a number of dependencies (MySQLdb, Numpy, PyQt, etc.) that it doesn’t necessarily need and it can be installed without them. If you don’t use the associated functionality, you won’t need these dependencies. But, irritatingly, ete2 tries …
Over the years, Apple has fallen in and out of love with Applescript, its "official" scripting language for MacOS. True, Applescript isn't going to go away any day now and true, it is a very simple language and easy to use. But if you don't want to have to learn …
In which an oddity in SQLAlchemy is spotted, and it turns out to be a bug not a misunderstanding.
The merge function in SQLAlchemy lets an object seem to get pushed to the database, but actually stores and returns a copy of the object. This is handy for when …
How to make a good XML solution for Python better.
Python was late to adopt XML and even then the libraries provided weren't as powerful or easy to use as those available elsewhere. Fortunately, with Python 2.5, a subset of the ElementTree library was incorporated into the standard library …
Ah, archetypes. How I adore thee. Other ways of constructing Plone content are as nothing before me. You are quick to develop, flexible in your approach, sensible in your defaults. But cryptic in your errors.
A lesson or two on breaking archetypes: First, not all fields accept all widgets …
How do you give a user a new password? Well, in the "User & Groups" control panel, you select reset password for them. What if they've lost or changed their email address? In the same place, set the new one. What if "that doesn't work"? You can assign them a new …
Matplotlib is cool. Did I say it was cool? I meant very cool. While principally a plotting library, it can be used for image manipulation and drawing.
Which is where our first problem occurs (as of the 0.8.4 version of matplotlib and possibly earlier). Should you try …
arx my-id 'A Cryptic Moniker <moniker@mysite.com>'
Set up your identity.
arx my-editor vi
Set your default editor for changing logs etc.
arx make-archive moniker@mysite.com--archive ~/Documents/Arx/Commits
Set up an archive.
arx my-default-archive moniker@mysite.com--archive
Make an archive your default.
arx my-revision-library …I had to install git on a Redhat server, without the benefit of a package managers, local repository or anything like a sensible configuration. Because "security".
Why? because git calls cur_easy_strerror which was only introduced in libcurl 7.12.0. RHEL3 has …
One of Plone's marked faults has been the lack of an out-of-the-box RSS aggregation product. Although there have been a number of attempts (CMFSin, CMFNewsFeed, CMFFeed), these are unfinished, quirky or only semi-functional. Fortunately, PloneRSS is a robust solution for gathering and publishing newsfeeds on your site. Unfortunately, installation is …
Whenever I get a cryptic error from Plone (which is all too frequently), I google to see if anyone else has it. This is me paying back into the system.
After fiddling with the schema on a type, adding it then failed …
While compiling some code with gcc (that had previously compiled and run without complaint under CodeWarrior), the following warning was reported:
SblNumerics.h:334: warning: 'typename std::iterator_traits<_Iterator>::value_type' is implicitly a typename SblNumerics.h:334: warning: implicit typename is deprecated, please see the documentation for details
The offending …
Try embedding something like this in your code:
<dl class="collapsible inline collapsedOnLoad"> <dt class="collapsibleHeader">Title</dt> <dd class="collapsibleContent"> <span tal:replace="structure python: here.printRequest(request, errors)"/> </dd> </dl>
and supply the necessary printRequest somewhere in your code or as a script:
def printRequest (self, request, errors …
Not that I'm picking on ElementTree or anything but ...
Normally XML documents will declare their character encoding in their opening tag. And, normally, this will be utf8. (This may be a standard.) So this is a common sight:
xml version='1.0' encoding='utf8'?>
An oddity in ElementTree is that …
Good news: you can get reliable and easy access to a proper, fully-featured database from within Python.
Bad news: you may have to tinker to get it to (a) install properly. And if not done properly, this may cause it to (b) not work properly.
Good news: Download a .dmg …
Pydee is new and nifty graphical frontend for using Ipython. Unfortunately, there's no monolithic binary package available for installing (at least not on the Mac), and so users have to install and compile all the prerequisites. Given the the number of steps and small but important details, it's easy to …
One of those weird things that always slips my mind and always seems less obvious than it should be.
Parsing a date (object) from a string representation in Python seems oddly neglected or cumbersome as compared to the rest of the standard library that surrounds it. (Witness the number of …
How do you get a Plone theme to uninstall and restore viewlets to the usual way. That is the mystery ...
In a theme I recently built, I moved or hid several viewlets around for artistic effect. The problem didn't show itself until I uninstalled the theme. The viewlets …
While developing a series of forms using CMFFormController, suddenly one of the components couldn't be found.
To recap (and serve as an example to those neophytes), CMFFormController structures your forms as a series of form templates or scripts with metadata files determining how execution moves between them. In this …
Some uploaded files (i.e. the Plone File content type) aren't editable after creation. The edit tab is visible, but clicking on it takes you to the That content does not exist error page. The file objects are well behaved in all other cases. Uploading new files showed that …
Add this to list of list of things I'm surprised I didn't know about Python. How do you use a percent / modulo ''%'' symbol in a string?
I had to do this the other day and it stumped me. How could it be that in <insert large number of years> I've …
After a long hiatus, we return with a traditionally opaque C++ error message:
error: request for member 'close' in 'out', which is of non-class type 'std::ofstream*'
which like all the best C++ error messages is long, detailed and completely unhelpful. It could be triggered by a number of things …
The process of upgrading a Plone site can be intricate. The 2.0.x to 2.1.x jump is no different. (Anyone who has followed the series of quirks and bugs in Agapow.net over the early part of 2006 will have seen the evidence.) While you could just …
Doxygen is a free tool for documenting code. With a single command it can generate cross-referenced HTML documentation from any C++ or Java code. Furthermore, if the code is commented in a particualr styled, Doxygen can leverage that to enhance the documentation. The below is a selection of the most …
This document was originally written in late 2003 / early 2004. The scene has changed some in the meanwhile, with various offshoots of Arch and Subversion gaining a lot of mindshare, and some of the URLs are probably now invalid. However, the broad conclusions still hold, and this is left here …
Archetypes Visual widget - the WYSIWYG HTML editor that shows up on Pages in Plone 2.5 - is pretty damn nice. I've had problems with Epoz and this is the first embedded web editor that makes me think about giving up on restructured text.
Getting it to work in your own …
While some people have reporting installing PIL on MacOSX without problems, there are legion reports that say otherwise. Frequently this revolve around PIL installing without JPEG support. There are a myriad of solutions (involving editing make files, configure flags, using fink to install dependencies, using package managers, casting goat entrails …
I've gotten used to manipulating members of Python objects with getattr, setattr etc. But a recent similar problem had me stumped. I had a class that I wanted to create a large number of similar behaving properties on (they would all manipulate an internal dictionary):
>>> f = Foo() >>> f.contributor = 'xyz' …
There are many reasons why the "multiply-defined" error can occur at link-time, most of them annoying but mundane: failure to include compilation guards on header files, the compiler finding more than one version of a file (perhaps a backup?) on the file-search path, lack of the extern qualifier and so …
There are no arguments to 'X' that depend on a template parameter, so a declaration of 'X' must be available
I have a C++ program of moderate complexity that I have to return to every 12-18 months to fix an obscure bug or add a modest festure. And every …
looser throw specifier for 'virtual Error::~Error()' ... overriding 'virtual std::exception::~exception() throw ()'
While compiling some code with gcc (that had previously compiled and run without complaint under CodeWarrior), the following fatal error was reported:
Error.h:83: looser throw specifier for `virtual Error::~Error()' /usr/include/gcc/darwin/3 …
Recommended extensions and add-ons for your Plone installation, including some advice and hacks.
In theory Topics can do just about everything you would ever need to do as regards dynamically aggregating content. In practice, they seem to be rarely used. (Neither of the three Plone books …
The crude "lets drop a print statement in" approach keeps being useful even with advanced dynamic languages. Here's a (slightly) improved version for ruby that pretty prints any number of passed objects and (importantly) where the print call was made from:
def dbg (*args)
print("dbg: #{caller()[0]}: ")
args.each …
Around version 2.2, Python rejigged its classes with some useful extensions. Unfortunately these enhancements have been explained so poorly that they appear in little published code.
One such enhancement is __slots__. An attribute of this name in a class restricts what attributes can be created in objects of that …