One of the things I miss about my previous non-static blog was that it would automatically display the time of the last update. Since I store my site documents in a way that preserves file modified dates and timestamps across the replication I started to poke around how Octopress handles the concept of updates to existing posts. A little bit of digging revealed that including “updated: ” in the YAML front matter would render a modified date time in the footer of the blog post. Armed with this knowledge I back tracked the process of populating that information. After multiple dead ends and learning a bit about the structure of Octopress, Jekyll and Liquid I hit upon a solution. In Octopress the date and updated html are provided from the post object as ‘date_time_html’ and ‘date_time_updated_html’ to Liquid for usage in the templates. These are provided to liquid in an _includes snippet that can be found in ‘source/_includes/post/date.html’.

With a little digging, it was obvious that neither of those keys were being generated in the original Jekyll source or by Octopress plugins (in the plugin directory). A little bit of Google foo, however revealed the octopress-date-format gem and exposed how octopress was creating those pieces of data. The gem that was being loaded for me was in the 2.x branch as I am running Octopress 2.x (This was a major pain point for me as I was pulling my hair out trying to resolve why the modules I was looking for didn’t exist or were different. It turned out that Octopress 3 is in development and I was looking at the source of the plugin for that. Oops! Doh!). This Octopress date-time plugin works by attaching to the init hooks for pages and posts and dynamically generating the the html content for both the original date and updated fields.

Because the plugin works by attaching to the init hooks and I could find no way to guarantee an order to what is called by the hooks, I had to scrap the plan to also attach to init and simply populate the field if it didn’t exist. Instead I went the route of creating a generator that retroactively visits all the posts, evaluates them for an existing updated field. If the updated field doesn’t exist it then evaluates the modified time on the source file and uses that as the updated value. Now, becuase all of the init hooks have already run, we need to finally trigger an update so that the octopress-date-time plugin does it’s magic (Otherwise nothing will get displayed, because the ‘date_time_updated_html’ field won’t exist). Instead of looking into how to trigger a specific hook, I simply looked into the source code for the date-time plugin and made the ‘hack_date’ call against the post to update those fields. The major negative for approaching updated values this way is that the evaluation for populating the fields produced by the date-time plugin will be run twice on every post that doesn’t have an updated field already in the YAML front matter.

Below is the UpdatedGenerator.rb source code that automatically adds an updated date to all my posts dynamically.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

require 'octopress-date-format'

module Jekyll
  # Auto set the updated YAML front matter if it isn't set and outside of the grace period
  class UpdatedGenerator < Generator
    def generate(site)
      # Loop through all posts
      site.posts.each do |post|
        # Only work on posts without an explicit modified date set
        unless post.data.key? 'updated'
          # Insert the file modified time
          updated = File.mtime(File.join(site.source,post.path))
          post.data['updated'] = updated
          #Fix the date formatting for the page
          Octopress::PageDate.hack_date(post)
        end
      end
    end
  end
end

If you use storage that maintains the modified time stamps on your source files, this will also work for you. (Hint! Git repositories do not maintain the modifed file timestamps and using this with them will cause indeterminite behavior.)

Turns out that when using a Linux FIFO to allow sending commands to a running application, that I inadverntently caused “IO spin” as the input FIFO was being queried by the spawned application. The end result was I was burning CPU cycles of a single core while trying to read input from an async input. So my application went from an average of 10% usage on a core to over 100% (The application was multithreaded). This is something I’d like to look into in the future to understand why what I did caused this particular outcome. Until then, I’ve resorted to not piping stdin from a FIFO and dealing with a moving console output as I try to enter commands.

Perhaps this will be something I develop a small program to do and limit how often it tries to read from the named pipe to reduce wasted cycles.

With the alpha release and the promise of a 6 week rolling release cycle, Rust is finally looking like a reasonable language to pour time into. While I’m mostly an application developer, Rust excites me as a potential language to use to implement libraries where I need greater control of the resource envelope and implementation details. My work on a perceptual image hashing system could potentially benefit from developing my custom concurrent hashing system in Rust and providing system binaries for the application. This is currently the most resource intensive part of ImageTools v2 and could benefit the most from limiting and optimizing how images are interacted with. Full Java/Scala objects to represent images are wasteful and bloat the heap as they’re loaded for only the time that they’re being processed. Once we’re done generating hashes and thumbnails, we only ever need to reference the original images when the user requests the original image. Outside of that all comparisons and logic focuses on the generated perceptual hashes.

There have been (at least a few times) that during the course of developing a personal project, that I’ve considered using a plugin style system to solve a particular problem. However, I have never designed such a system and almost always find that what I wanted to do can be solved in another way that is far simpler. A few days ago I discovered a real use case for such a pattern and finally bit the bullet and dived into discovering how I could go about implementing such a pattern. Because the project was in Ruby, I hoped that the solution would be simpler than if I tried to implement it in a stricter language.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

# Class that will dynamically be altered to reference our plugins
class PluginConfiguration
    # A static list of loaded plugins
    @@loaded_plugins {}
    def get_loaded_plugins
        # Allow access to loaded plugins from instances
        @@loadedPlugins
    end

    # Simple not_implemented method for default example
    def not_implemented
        raise "Not Implemented"
    end

    # "load" the default into the listing so we can call it dynamically
    @@loaded_plugins['default'] = :not_implemented
end

The above code is the guts of the plugin system I’ve been developing. Each plugin system will need an appropriately named plugin configuration object to store information about the loaded plugins and also serve as the object that is modified with the contents of our plugins.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

# The interface to the plugin system
class PluginManager
    attr_reader :default_plugin
    def initialize(default_plugin = 'default')
        @default_plugin = default_plugin
        @config = PluginConfiguration.new
    end

    # Allow access to the list of plugins that are recognized as loaded
    def get_loaded_plugins
        @config.get_loaded_plugins
    end

    # Interface to call for the plugin to act on some sort of data
    def process(plugin_name, data)
        # Reference to what method will by handling our request
        plugin_handler = nil
        # Check to see if the requested plugin is loaded and if so, use it
        if get_loaded_plugins.include? plugin_name
            plugin_handler = @config.method(get_loaded_plugins[plugin_name])
        # Otherwise use the default
        else
           plugin_handler = @config.method(get_loaded_plugins[@default_plugin])
        end
        # Call the handler and return the result
        plugin_handler.call(data)
    end

    # Dynamically call for the loading of all plugins in the ./lib/plugins directory
    Dir.glob('./lib/plugins/*.rb').each{|f| require f}
end

This is the guts of the plugin handler and acts as our interface to calling loaded plugins. The “process” method is used to route the request to the loaded plugin or gracefully fall back onto the default handler which can do whatever we want. In this case I simply have it raising an error that what we’re trying to do is not implemented.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# Module that will be the basis for our processing
module NewPlugin
    # Method that we'll register to act for our plugin
    # It's a good idea to prevent the stepping on other plugins to name the methods according to the plugin name
    def new_plugin_process(data)
        # Do the work as the plugin
        puts "Called new processor plugin"
    end
end

# Overload the PluginConfiguration class to include our new plugin
class PluginConfiguration
    include NewPlugin
    # Set the plugin in our registry so that we can call it dynamically by the name set
    @@loaded_plugins['new'] = :new_plugin_process
end

This is the implementation of a simple plugin that will, when loaded, insert itself as a module in PluginConfiguration and then register as a callable method in the loaded_plugin hash. The biggest issue here is to make sure we won’t step on any other loaded plugins.

The idea behind this system is that we use mixins to dynamically increase the functionality of the PluginConfiguration class, and then we register that functionality into the hash so that it can be dynamically called. This way we have runtime dynamic methods without having to define them in the code. My use case that led to this design was a dynamic parser that can be configured to parse a source differently based on the expected content. For example an RSS feed can have a special RSS parser, and Atom feed can have an Atom parser, raw HTML can have it’s own parser, and any other source we can dream of can have a parser. As I work, I can write the parsers independently without having to update the plugin manager with the list of loaded parsers.

I’m sure there are plenty of other ways to do this in Ruby, and my way is probably not the best. But it is working wonderfully for what I need it to do.

Since I am starting back up and writing about some of my exploratory programming, I thought it only fitting to greet everyone in the languages I’ve been using recently.

1
2
3
4
5

class HelloWorld {
    public static void main(String[] args) {
        System.out.println("Hello World!");
    }
}

Hello World!

1
2
3
4
5

object HelloWorld {
    def main(args: Array[String]) {
        println("Hello World!!")
    }
}

Hello World!!

1

puts "Hello World!!!"

Hello World!!!

1

print "Hello World!!!!"

Hello World!!!!

1
2
3

fn main() {
    println!("Aw C'mon")
}

Aw C'mon

I’ll be posting interesting things here from now on. They may not all relate to programming or technology, but I imagine most of them will.