The PictureLibrary is a meant to be a browsing and organizational tool for pictures. The difference between this and many other picture orgainizational tools is that allows the user to create their own logical heirarchies and place pictures in one or of more of the categories1 without imposing restrictions on how or where the images are located within the user’s file systems. The rest of this part is dedicated to describing the operational concepts from various perspectives or user concerns.
One aspect of this tool is that all of the pictures should be browsable even when the images are not present on the local file system. For instance, I am limited to just the last years worth of pictures and the rest are kep on removable media. The always fully browsable aspect generates three requirements on the tool. First, what ever is stored about the image should be smaller in size than the image itself othewise just keep the origninal images around2. Also, all of the information saved must be done in a persistant way meaning that the saved information must be accessible when the original images are moved to a removable or remote file system. Third, all of the data must be quickly retrievable for easy browsing and searching.
Second ascpect of the tool that needs to be discussed is whether it should have a GUI or be a collection of scripts. My sense of this choice is to make it a GUI and not worry with the command line because we view images in some form of a GUI anyway. I would also like to tie it strongly to GIMP http://www.gimp.org/ for tracking changes and heiredity3 of a picture. In the end, however, it would be better to keep the front-end (GUI and/or command line) separate from the logic of how the data is stored and organized because I am not the best GUI developer and how that is built is more likely to change than any other part.
The idea of the tool is that the user has some set of images that potentially have some metadata associated with them. We may then want to organize them in some way to make it simpler to browse in order to find a specific picture. Searching is typically not terribly difficult when the pictures are strongly organized within the file system, although binding user data with images with just the file system can be trickier. What if we want that picture of the lake where the trees were reflecting so nice and it had the moon in it too. Searching for that picture would be difficult if not impossible with just file system organization. Browsing for it could be time consuming plus error prone due to the potentially high number of images one would have to look at4. For instance, let us say that we have lots of images and want to be able to find all of the pictures of the flower California Poppy because someone wants to see the one you showed them three years ago. If you were lucky, your file system structure would be adequate to handle the browsing. However, it is more likely you would have to look at lots and lots and lots of pictures that were not even of California Poppy. What would be nicer than the file system is a logical strucure or many logical structures that you could define at any time and add one or pictures to them at will. Of course, that is the point of this tool.
I have already mentioned that the categories form logical hierarchies (trees) that allow the user to organize their pictures. Lets go back to our example where we have lots of pictures and add that we took them at places like Denali National Park, Mount Raineer National Park, and Yosemite National Park. Lets also constrain the pictures a litle more by saying that they are mostly scenery and flower pictures. Now, to continue with the California Poppy and reflecting lake examples we may want to create a set of groups to put these pictures into like Landscape and Wildflower. However, Landscape could be further divided as could the wild flowers. It may be of interest to not only file the flower pictures under their common name but also their scientific name. Filling in more the of the details, we could end up with a tree that looks something like the one shown in Figure 1 on page §. In that case, all of the pictures would be found in more than one category. The user created hierarchies then give the user more freedom of organization than the file system alone.
Categories should be able to be bound as well. In other words, it might be more satisfactory to file flower pictures into its specific species name under Plantae and then link the common names under Wild Flowers to the correct set of species for that common name. The link should be uni-directional which means that selecting the common name will show all of the pictures that are filed in the categories it is bound to, but selecting one of the bound species categories will not show the pictures in the common name. Of course, this means the user must be able to bind and unbind these links.
In UML the same would be said with a use-case as shown in Figure 2 on page §.
Metadata is defined as name-value pair tuples with all names being unique. EXIF http://en.wikipedia.org/wiki/Exif information is already of this form. While their names are encoded in integer values, the idea is the same. In addition to EXIF http://en.wikipedia.org/wiki/Exif information, the user may want to attach metadata to a picture. For instance, I may attach information about the hike to the picture to remind myself of why I took it. I may also use log entries to track idea of why I chose to make the modifications that I did. It does not take too much imagination to think of lots of information that would be nice to attach to a picture.
A second aspect of metadata that would make it more convenient is to group the different types. In other words, it would be nice to no have the hiking information intermixed with the EXIF http://en.wikipedia.org/wiki/Exif information or the picture modification log entries. To this end, metadata can then be grouped with the side affect that every name in a group should be unique rather than globally unique and each group name is unique.
Since the non-EXIF http://en.wikipedia.org/wiki/Exif metadata is defined by and for the user, it must be editable. Adding metadata information should aid the user in that all names defined for a group should be easily accessible to the user while defining new names should be easy as well.
As already defined, the picture is more than just the image. It is also all of the metadata, from the user and file system, associated with the image as well. There is not much to say beyond this.
I have modelled the physical location after my own work flow and file system structure. To start with, I collect all of my images in two directories: Pictures and SlideShow. The Pictures directory holds all of the raw images while the SlideShow holds all of the modified images from Pictures that are worth showing to anyone. When the disk nearly full at the end of the year, I carve the Pictures directory into DVD size blobs and place the SlideShow directory in each blob. The idea of putting the SlideShow directory on each DVD is to make it easier to search. For the first thousand pictures or so it worked just fine. Getting up around ten thousand pictures and five years of photos the organization method is starting show signs of the stress. Also, it provides no way to search for all pictures based on lens, focal distance, or any other image metadata.
Because of the above work flow and file system structure, I have chosen to break the physical location of an image into two distinct parts. The first part is the root location and the second part is the relative location to the root. The two elements combined make up the concept of the physical location of an image. The root location can be changed without changing any of the relative locations so that when DVDs are made, the root can be changed without changing anything of the relative locations.
A restriction from this concept of a physical location means that images cannot be added to the library in an ad-hoc way. I am not sure whether I like this restriction or not. However, it exists because I would rather have the process of moving data to DVD simpler than have the ability to add images willy-nilly. Another restriction, but a beneficial one, is that the root locations should be scanned for images rather than have the user add them.
Part I covers a short story board of what I thnk the PictureLibrary tool should do. In Part II, we will use that story as foundation and rewrite it into engineering terms: scope, operational concepts, and requirements. The scope is composed of a single need, many goals, one or more objective5 per goal, and possibly a short rationale. The operational concepts are a much more complete version of Part I. The requirements are those horrible one liner thou shalt statements that I hopefully will never see the need to write.
To define scope and requirements, it is important to first define the need the tool must satisfy: The PictureLibrary satisfies the user’s need to quickly locate a set of specific pictures amongst the many that they own.
Statement The most important goal, at least for me, is to make sure that the PictureLibrary tool does nothing more than aid the user in organizing their pictures.
Objective No architectual, design, or implementation element should specify any internally developed functionality beyond the orgainization of pictures.
Rationale All too often picture organization tools start to drift into image editing because the picture is right there in front of the user. Well, I do not want to spend my life re-writing GIMP or ImageMagick so my primary goal is to constrain the scope of the tool to just organization.
Statement The PictureLibrary tool should not depend on any OS specific functionality.
Objective No architectural, design, or implementation element should call out or require any specific OS or OS component.
Rationale The PictureLibrary tool is supposed to be an organizational tool only (see Paragraph 7.2.1 on the preceding page) above and beyond the ability of the OS and file system alone (see Section 2 on page §). For instance, Linux and file systems allow for symbolic linking of files allowing for an easy way to implement many of the features described in Section 3 on page §. However, the Windows OS and file systems do not allow for symbolic links. Therefore, depending on the OS to have symbolic link capability would limit the tool. However, being able to handle OS special cases is a necessity. For instance, even though the tool should never depend on symbolic links being present, it should handle them logically rather than crashing or misbehaving.
Statement Configuration, organizational information, and generated data should persist between executions, removal of original images, and computer power cycles.
Objective Reboot the computer or restart the application and see that the organizational data is still present.
Rationale It does not make sense to require the organizational information to be redone at every instance. I have 10000 pictures and I do not want to have to sort them every time I start the tool. Many people store their images on removable media such as DVDs or CDROMs. It seems appropriate that the tool should be let the user browse and search those pictures as well as those on the local file system. All together, it means that the tool should have a great deal of persistence.
Statement User can define heirarchical structures.
Objective No architectural, design, or implementation element should specify any part of the category tree structure or names with the exception of the tree’s root node.
Rationale Section 3 on page § is the best explanation for why it is important that the organizational structure remain the pervue of the user. It would not be consider in conflict with this goal to offer quick buttons that create commonly used heiracrchies. However, those structures should not be deemed immutable or any more rigid than any other user defined structures.
Statement Pictures should be represented with thumbnails and with some user defined sized image.
Objective Should be able to view any image in either a thumnail or a some user defined size.
Rationale Thumbnails are great for lists and getting a quick idea of what a picture is, but they lose a lot of information. The user defined size image would be one at a time view that retains a fair amount of detail from the original image. Since all pictures are not created equal, the best size for retaining detail but reducing its footprint is best left to the user.
Statement How the user interacts with the tool should not affect how the organizational information is structured.
Objective The architecture should specify dependencies that ensure how the organizational information is structured is independent of the GUI.
Rationale While this borders on an architecture decission, I think it is important to at least state up front that how the data is stored, organized, and managed should be independent of how it is viewed and how it is altered.
Statement Should not store duplicate entries to keep data storage at a minimum.
Objective The design should specify some hash for uniquely defining an image.
Rationale Part of the idea in creating this tool is to reduce the amount of disk space assigned to holding pictures. Duplicate information would seem to instantly break the idea of reducing the disk resources assigned to holding pictures.
Statement Should work with removable file systems as well as local and remote ones.
Objective Pictures to be organized should include images from removable devices like a DVD but those images should be viewable even when the DVD is not present.
Rationale Many users, myself included, use removable WORMs6 to archive pictures. The archived pictures should exist in the PictureLibrary tool and be browsable and searchable even when not mounted. I personally have a dozen or more DVDs and one DVDROM making it hard to browse all of those DVDs at once. The PictureLibrary tool should allow the user to browse and search all of the archived pictures even when the archive media is not present.
Statement Should give the user quick and easy access to some image editing tool.
Objective Given the user has selected a picture(s), they should be able to open it(them) with their favorite image editor.
Rationale While I have already stated that the PictureLibrary tool should not venture off into image editing or file system manipulation, the motivation for the PictureLibrary tool to drift into that realm is because getting to images should be very easy. In other words, the user has just found those three pictures that their friend wanted prints of and now would like to open them in their favorite image editor to do the final curves adjustment and then print. Well, rather than do a poor job at that in PictureTool, let the user define thier favorite image editor and send the pictures to it.
All of the following operational concepts assume a GUI. I will try to keep the operations based on a general GUI like context menus instead of right click so that the GUI can be developed independently of the platform. Also, I will try to keep OS references independent of platform as well. However, I will probably goof along the way so know that I run Linux (Ubuntu) and using Python for the language, WX for the GUI, and Postgresql as my database for persistence and relations.
The first four concepts, Subsections 8.1 through 8.4 inclusive, are expected to be the most common operations. The remaining Subsections are for maintenance of the organization, information, and persistance. Much of the information would be repeated for startup and shutdown of the program. Instead, the process of starting and stopping the tool are covered here while the Subsections are encompassed between.
I gotta put some very serious thought into this one. I may end up pulling it depending on its difficulty to define.
All of the operations that can be performed on an existing category should be selectable through a context menu. Adding root categories may be selected through a root menu, context menu, or both.
The intent of this part is to take the information in Part I and translate it into a software structure with the intent of this archetecture is to separate the GUI toolkit, database, and image reading routines from one another allowing people to develop different elements to suit their needs or desires while maintining all of the core functionality.
The overall architeture is a Model-View-Controller implementation and is shown in Figure 3 on page §. The CONTROLLER handles the user generated events and changes them to calls for data changes within the MODEL. The VIEW displays the data contained with in the MODEL and keeps pace with any changes. The MODEL represents all of the pictures and their organization.
The CONTROLLER can be decomposed into somewhat smaller componets as shown in Figure 4 on page §. In this decomposition, the EVENTLISTENER is shown as the upper most unit and is responsible for handling the events generated by the VIEW. The middle teir is composed of all the commands the CONTROLLER must process. The bottom teir consists of the translation to calls into the MODEL and WINDOWEVENT.
The VIEW is decomposed into a MODEL listener and then all of the visual components, as shown in Figure 5 on page §.
The decomposition of the MODEL, shown in Figure 7 on page §, is probably the most significant to the overall operation of the software. It is the MODEL that needs to define listeners and call “interfaces” to isolate the GUI from the data management. It is also the portion that keeps the structure as seen by the GUI independent from the structure that exists in the database. Lastly, it is the portion that will define all of the available image reading routines.
The database is constructed with 13 tables and shown in Figure ?? on page ??. There are three data tables, four hybrid tables, and six indexing tables. Let me define the different table types:
As is probably normally the case, no table should ever contain duplicate records.
The first and most obvious table is the one that contains the fundemental picture data as shown in Table 1 on page §. The field md5 is used to contain the unique identification value for a picture. In other words, any to images which generate the same MD5 checksum must be the same image7. The mini entry contains a user defined sized image for viewing in the tool and the thumb entry is for the thumbnail. Typically I find that thumbnails are great for a general idea, but the mini is to be used for a more detailed view than 128x128 pixels brings. All of the metadata then associated with any given picture is referenced though any of the various indexing tables with the pid field.
The next obvious table is the name of categories the user defines and is show in Table on page §. As stated in Section on page § the categories are a heirarchial and its therefore necessary, at a minimum, to track every category’s parent, which is the reason for the pcid entry. The name entry holds the actual name, and the cid entry is used for indexing.
The metadata then requires three tables. The first, Table on page §, records the name of the name-value pair8 while Table on page § records the value. Keeping these two tables separate is a bit of an arbitary decision on my part. The question is really one of how many bytes are the names and values going to be. If, on average, they are less than 8 bytes, then these tables should be combined. However, I am a very verbose kind of person, and I would be surprised if my names and values, again on average, are less than eight bytes each. Therefore the tables are separated to save space. The last, Table on page § is to place names into groups. For instance, the “F Number” name should always show up in the EXIF group while the “Comment” name should always show up in the LOG group where EXIF and LOG are defined by the user. In some sense, the groups are a flat set of categories for metadata.
There are only two hybrid tables and they deal with where the image of a picture resides. The first, Table on page §, describes a root location where many image files are found. The root location is then mapped to a category. The choice to do this allows the user to assign a logical name (category) to a DVD or CDROM or hard drive location. Hence, when the user is browsing the photobook, they see the logical name for the location like “2007 Disc 2 of 3” instead of the physical mount point of “/media/cdrom/”. The other hybrid, Table on page §, is the image location relative to the root directory. Since it is a relative path, it reference which root location it is associated with.
Once possible benefit of the root location as described is if the pictures are to be moved en-masse. In other words, if the user wants to burn the directory and its structure “My Pictures” to a DVD then only the root location would have to change. Whether or not this is reasonable is a very debatable point. I personally see it as a marginal, at best, feature because when archive my pictures to DVD it is always more than one DVD at a time.
Table on page § relates a picture and category or files a picture. Records are considered the same if the corresponding cid and pid fields are the same values.
Table on page § relates the metadata to a picture. Records are considered the same if the corresponding pnid, pvid, and pid fields are the same values.
Table on page § allows one picture to be defined as the parent of another. Records are considered the same if the corresponding pid and ppid fields are the same values. Without tight integration into the user’s photo editor, this entry has very marginal utility. However, it is a utility I greatly desire so I am keeping it in.
Table on page § allows the user to logically link one category to another. For instance, the user may have a whole structure of the scientific names of flowers and another tree of their common names. In this case, it would make sense to link the common name Poppy to all of the species that we call Poppy. All links are unidirectional meaning when viewing the category Poppy then those pictures in Escscholzia californica subsp. californica and Escscholzia californica subsp. mexicana are visible but when in Escscholzia californica subsp. californica or Escscholzia californica subsp. mexicana the pictures in Poppy are not visible. Records are considered the same if the cid and aka fields are the same.
Table on page § relates where a picture is located. The main reason to use an indexing table rather than combine this information with the picture in Table on page § is to allow the same picture to exist in more than one place at a time. Records are the same if the lid and pid fields are the same.
Configuration is so trivial I am skipping the design documenation for it.
The purpose of the Model API, shown in Figure 8 on page §, is to allow different Views and Controllers to be used against a single Model to reduce data corruption. The FUNCTIONS class allows external entities, in particular a Controller, to stimulate the model and cause changes. The OBSERVER_API class allows external entities, in particular a View, to be notified when changes occur within the model. The data types that flows in and out of the Model are represented by the CATEGORY, GROUP, LOCATION, METADATA, PICTURE, and ROOTLOCATION classes.
The model completely encapsulates the storage and persistence of the model, from the Controller and View. However, to keep the tool at least mildy efficient in the use of memory it would be nice to lazily read the data from the database. In other words, do not read in the mini image if it is never needed. The hidding and lazy reading is done using the CONCEPT class and its children shown in Figure 9 on page §. From Figure 9, we see that CONCEPT depends on the DB.ACTION_API to access the data and from inheritence it can be derived that the children may depend on the API as well. The relationship among the classes realizes the earlier definitions and concepts from Section I on page § in that a Picture is made up of a single IMAGE, a set of GROUPs, and a set of LOCATIONs. The METADATA assigned to an IMAGE is accessible through the GROUPs.
The purpose of the database API is to separate the database access code from the rest of the software system. The database is very low in the system and parts of the system can accidently become coupled to a particular database and its implementation if care is not taken to avoid it. Figure 10 on page § is how the API is subdivided. The two interfaces, ACTION_API and TYPE_API, are the units that keep the unique portions separate from the general portions. The class FUNCTIONS is responsible for converting database concepts and practices to internal representations that are dependent on the language and libraries being used.
The purpose of the Image IO API is to allow simple extension of the image reading abilities of the tool. For instance, PIL can read many image types, but more can be read through ImageMagick and even more through GIMP. However, not everyone requires all these options and may not want to install everything necessary to get them. So, the Image IO API was designed to encapsulate the reading of the images and exif data from the Model. Since all extensions are required to implement the IMAGEIO.ACTION_API, the Model can then loop through all of the available options and try them on each file until one of them succeeds. The only reading utility provided with the tool by default is PIL, which should be considered the reference implementation.
The design for the image IO encapsulation is show in Figure 11 on page §. Here we can see a FUNCTIONS class that allows the Model to loop through all available ACTION_APIs until the image is read. The image is then processed and returned in an instance of IMAGEINFO_API. Memory efficient use when reading many images needs to be taken into consideration in the implementation portion of the FUNCTIONS class. Since the ACTION_API handles a single filename at a time, there is nothing it can do to improve effeciency.
There is no special design for testing. The only thing to say about test design is that it should test all of the _API classes and exercise as much of the Controller, Model, and View independently as it can. Code coverage is very important to the testing.
Normally I would implement such a too in Java, but I have chosen Python to develop and deploy with for two reasons. First, I like the different libraries for reading the varying image formats. I would like to use PIL by default and internally, but it should be easy to add the ImageMagick library or some library to read raw formats as well. Second, for using different GUI libraries. I like the richness of Wx, but it is not supported on every platform; for instance 64 bit Windows Vista. Therefore, it would be nice to have a TkInter and other GUIs.
I have two major reservations with the Python choice. First, the implicit typing is terrible. I know everyone calls it dynamic typing but that is being over gracious. When any function is defined and implemented it has an expectation on the type being passed in as the argument. The fact that these expectations are not declared in the function prototype but are declared in the comments, if they are declared at all, makes the expectations of the type implicit. The data types passed in do not mutate to a some reasonable expectations within a function, which is what dynamic types would do. In any case, this implicit typing makes it much harder to develop a well structured piece of software. Second, Python lacks the software engineering tools – mostly because of the implicit typing – that languages like Java and C/C++ have. I am very used to applying static analysis tools, software structure management tools, and others to projects to maintain the desired architecture and design. All of these tools seem to missing with Python or are not very effective because of the nature of the language. In the end, the multiple image libraries out weighed my concerns and I chose Python, but it comes at the cost of a much more disciplined development style.
Since I have chosen Python, all software written should attempt to follow the Style Guide http://www.python.org/doc/essays/styleguide.html. Of course, the styles are so weakly defined and pretty much the whim of the author that I am taking some liberites on the style. First, I want to deploy the tool in a Python Egg http://pypi.python.org/pypi/setuptools#using-setuptools-and-easyinstall making heavy use of the Python package and classes. Package names will be CapitalizedWords unless it is an acronymn in which case it will be all upper case. Module names should always be a single word and lower case. Classes in a module should always be CapitalizedWords. All methods within a class should be mixedCase, begin with a single underscore for protected and double underscore for private where protected and private hold the C/C++ definitions. The major element missing from Python is the Java Interface. All classes meant to be an interface should end with _API and all of the methods should throw a InterfaceMethodNotImplemented exception as its only functionality. Unfortunately, pushing the exception to run-time means that problems will become harder to solve, but it is the best that can be done. Lastly, all class variables should be private and mixed case. Other than configuration items, there should be no global variables and the configuration items should be lower case with underscores.
I have also attempted to put information into the doc-strings. I have tried to keep the information between this document and the doc-strings distinct. I am using this document to detail why I made decision that I have and the doc-strings to document what is going on. I am sure that there will be overlap in the end and both will be stale relative to the source itself.
This is a particularly simple package. It as one module called access that contains the two functions read and write. Both read and write accept a simple file name as the location of configuration data. To read the file, it is simply executed. To write the file, the write function takes all of the configuration data and writes to the file using legal python syntax so that it can simply be executed.
The construction of the package Configuration should allow the developer to access all of the configuration items directly; e.g., Configuration.my_parameter. All parameters should have a default value assigned to them when the package is initialized for the first time. Doing so make older configuration files compatible that have missing parameters with newer versions of the program. Configuration items should have very explicit names as well that include the package, module, and class they are associated with to prevcent name collisions; e.g., Configuration.picturebook_view_wx_cat_sash_position. It is very important that they all start with ’picturebook_’ too!
The tables are defined in the ’Python module PictureLibrary.Model.DB.tables’. They result in the schematic show in Figure 12 on page §. The actual schematic can and should be checked frequently. To do so, create a test database using postgresql (’createdb test’) and run the system test modules. They should leave a database in place that ’postgresql_autodoc -t dot test’ can be run against to generate a dot (Graphviz) document. Then use ’dot -tps -o test.ps test.dot’ to generate a poscript file that shows the pretty image.
The Figure 12 shows that the database architecture from Section 11.1 on page § has been met. We see that there are five data tables that everyone else references, two hybrid tables that index as well as contain unique data, and then seven indexing tables.
Figure 10 on page § shows the conceptual separation. Given the language is Python and it contains the DBAPI 2.0 specification and implementation, FUNCTIONS will be implemented as a module while all of the other units will classes. The first and reference implementation of the ACTION_API and TYPE_API will be using postgresql.
The postgresql implementation is now done. Its completion has defined how new databases can be supported without much any modification. First, the package containing the new database implementation must PictureLibrary.Model.DB.XXX where XXX is the user’s name for the database implementation. For postgresql I chose the name Postgresql making the full package name PictureLibrary.Model.DB.Postgresql. Then, within the __init__.py for the XXX package, the following function must be defined: getActionInstance(). The function must return an instance of PictureLibrary.Model.DB.Action_API. Of course, PictureLibrary.Model.DB.Action_API must return a PictureLibrary.Model.DB.Type_API so the user must implement that interface for XXX as well.
The ImageIO package is implemented with a similar plugin ability to that of the DB. When the main program starts, the configuration is loaded and the picturelibrary_model_iio list is defined. It then calls PictureLibrary.Model.ImageIO.registerPlugins() to iterate through the configured list and load all of the libraries.
If the developer wants to add a new ImageIO library to the tool, they must first start with the package PictureLibrary.Model.ImageIO.XXX. For instance, PIL and GIMP are already supported and their package names are PictureLibrary.Model.ImageIO.PIL and PictureLibrary.Model.ImageIO.GIMP respectively. The __init__.py for each of these packages contain the signle function getActionInstance() whcih returns the implementation of PictureLibrary.Model.ImageIO.Action_API. Hence, adding new ImageIO libraries should not be overly difficult.
The implementation of testing is done with python doctests for the _API implementers and then a program for covering the Controller, Model, and View. I wanted to use the pyunit for some of the tests, but it proved too difficult with the model because a database is required and the tests have to be performed in a certain order, which is conflict with the concept of unit testing.
The hardest part about constructing GUI software is separating the control from the view. It seems odd to me that this should be the case given that the Model-View-Controller Pattern has been around since the dawn of desktop, but I have been giving it a try anyway. In this attempt at decomposing the problem, all user input is handled by the Controller while all of the model changes and simply view changes are kept internal to the View. In terms of implementation, it means that the Controller pretty much registers for all of the events the View generates. Because the Controller binds itself to the View’s events, the Controller becomes bound and intimately coupled with the View. Not really what I wanted, but better than the alternative.