learning tech · Programming · Python · Uncategorized

Ding Ding Ding

A constant barrage of links. Each one vital for me to read but I can’t read them now. That’s where my little application I’ve written comes in. It’s also how I learned some very cheeky coding.

Take a look at the source code: https://github.com/geekgirlbeta/linkapp

The application allows a user to make a post with a url-link, title, and description. Upon my first pass of this application I made the choice to use Mustache templates in my HTML. I won’t get into the why of it in this post, but seriously, it’s called Mustache.

Onto the problem: the application has a form which allows the user to make a post. Once all the required fields are populated in the form the wsgi app sends the collected data to a manager object to ultimately be stored in redis as a hash. It was after I had all that completed and had made the choice to use Mustache that I wanted to add a tagging feature. Part of this feature would allow a person to click on a tag within a post and all posts with the same tag would be listed. I encountered a problem adding this feature because I was using Mustache. I populate the link in the database with a string of tags (separated by the pipe/vertical bar symbol). Using the string directly, I would not be able to loop through the tags with Mustache to display links for each tag.

Redis-py has a method called set_response_callback(). With it, a custom callback can be added to the Redis connection on a per-instance basis. It takes two arguments, a command name and the callback.

import redis

def my_callback(response, **options):
    # do stuff here

conn = redis.StrictRedis()
conn.set_response_callback('HGETALL', my_callback)

With this method I was able to write a custom callback to massage the data for use in mustache.

Redis returns a list containing each key followed by its value:

['author', 'admin', 'page_title', 'Testing Number Two', 'tags', 'new|test|tags', 'desc_text', 'this is the post desc for number two', 'created', '01-08-2017 @ 20:50', 'key', '6123456210389ca5906a2a721a9ed793', 'url_address', 'http://notarealurl']

By default, redis-py turns this into a dictionary. I then needed to split the tags into a list. For Mustache, however, I needed to expand the tags into a list of dictionaries:

[{'name': 'python'}, {'name': 'learning'}, {'name': 'tutorial'}]

To make this easier, I made a LinkWrapper convenience class for wrapping the raw Redis data for Mustache use. Since Mustache can use a dictionary or an object, not only am I able to process the tags, I am also able to take advantage of the object oriented features of Python. That’s covered later in the post.

I called my response callback hash_to_linkwrapper(). It takes the Redis response list, the result from HGETALL, and returns a LinkWrapper object that can be used in Mustache templates in place of the dictionary that redis-py returned previously.

The hash_to_linkwrapper() function uses three built-in python functions. The idea came from the redis-py source code, in the default response_callback for the HGETALL command.

When I first saw that code, I didn’t fully understand what it was doing. So I dug into the python documentation, and did some experiments in my interpreter. Here’s what I’ve learned:

Redis-py uses Iter(), which returns an iterator object that wraps the response list of key-value pairs. The documentation states if there is no second argument that the object must be a collection object. The response list satisfies that requirement.

The resulting object is passed to zip(). Zip() returns an iterator of tuples. We want a list of two-tuples containing each key-value pair, that we can pass to dict. Passing the iter() object to zip twice accomplishes this, but it wasn’t clear exactly how iter() and zip() were working together, at first.

Dict(), creates a new dictionary. It can use a list of two-tuples to do so. Redis-py passes the iterator of two-tuples zip() creates to the dict() constructor.

Putting it all together, here’s my hash_to_linkwrapper callback:

def hash_to_link_wrapper(response, **options):
    if not response:
        return {}

    it = iter(response)

    attributes = dict(zip(it, it))

    return LinkWrapper(**attributes)

After digging into how all of this works, I now understand what iter() and zip() are doing together. When you pass the same iter object to zip() twice, it takes the even items in the iterator and use them for keys and the odd ones for values. The reason this works is because the same iter object is passed both times. When looped over, an iter object returns the next item in the list passed. Since the same object is used twice, the list we passed to iter() is consumed in half as many calls. This is equivalent to slicing the list into even and odd sets, or looping over the list with a for loop and sorting the keys from the values.

hash_to_link_wrapper() then turns the key-value two-tuples into a dictionary, which is used as keyword arguments to the constructor of my LinkWrapper class. I can now use that object with mustache.

Here’s the finished callback and LinkWrapper class:

class LinkWrapper:
    """
    Convenience class for wrapping raw redis data for Mustache use
    """
    def __init__(self, tags='', **kwargs):
        self._tags = tags

        for key, value in kwargs.items():
            setattr(self, key, value)

    @property
    def tags(self):
        """
        Making it easy to get a list out of the tags property.
        """
        return [{"name": x} for x in self._tags.split("|")]

def hash_to_linkwrapper(response, **options):
    """
    Takes a redis response list (result from HGETALL) and returns a LinkWrapper
    object that can be used in Mustache templates in place of the typical
    dictionary that redis-py returns.
    """
    # if the response is false, an empty string or empty list, etc,
    # return a dictionary - this is OK for mustache
    if not response:
        return {}
    
    it = iter(response)
    
    # SUPER CHEEKY
    attributes = dict(zip(it, it))
    
    # make each member of the dictionary into a keyword argument to pass
    # to the LinkWrapper constructor.
    return LinkWrapper(**attributes)

There were other ways I could have handled this problem. I could have not used mustache to begin with (lol), but once it was in place it was better to find a solution to make it work rather than to abandon it. Goonies never say die. 🙂

Instead of using iter(), dict(), and zip() I could have used the following:

def hash_to_linkwrapper(response, **options):

    if not response:
        return {}
    
    attributes = {}
    
    for index, value in enumerate(response):
        if index%2 == 0:
            attributes[value] = response[index+1]
    
    return LinkWrapper(**attributes)

If I wanted to use this code without the LinkWrapper I could do this:

def hash_to_linkwrapper(response, **options):

    if not response:
        return {}
    
    attributes = {}
    
    for index, value in enumerate(response):
        if index%2 == 0:
            if value == 'tags':
                attributes[value] = [{'name': x} for x in response[index+1].split('|')]
            else:
                attributes[value] = response[index+1]
    
    return attributes

One of the reasons I created the LinkWrapper class was to take advantage of lazy loading. Lazy loading defers the processing of some data until the last possible minute when it is needed.

This might seem like over kill at this stage of the project, but coding it this way in the application lets us be open to more object oriented code in the future.

Some examples include:
similar massaging of other data (e.g, capitalizing the page title)
validation – by implementing the setter for certain attributes we can raise an exception if invalid data is given.
presenting a more structured data model to people using the backend API.

If we need to present specific values to Mustache, we can extend a simple Link class, instead of hard-wiring things into our WSGI code.

Here’s a quick example of where we can head in the future:

include strings
class LinkTitleMissingError(Exception):
    pass

class LinkTagsEmptyError(Exception):
    pass

class Link:
    def __init__(self, tags='', page_title='', **kwargs):
        self._tags = tags
        self._page_title = page_title
        
        for key, value in kwargs.items():
            setattr(self, key, value)
    
    @property
    def tags(self):
        return [x for x in self._tags.split("|")]
    
    @tags.setter
    def tags(self, value):
        if not value:
            raise LinkTagsEmptyError("At least one tag must be provided")
    
        if "|" in value:
            value = [x.strip() for x in value.split("|")]
        elif isinstance(value, str):
            value = [value,]
    
        value = set(value)
    
        if not value:
            raise LinkTagsEmptyError("At least one tag must be provided")
    
        self._tags = "|".join(value)
    
    @property
    def page_title(self):
        """
        Upper case the page title
        """
        return string.capwords(self._page_title)
    
    @page_title.setter
    def page_title(self, value):
        """
        Validate the page title
        """
        if value is None or value is "":
            raise LinkPageTitleMissingError("Page title is required")
    
        self._page_title = value

class MustacheLink(Link):
    @Link.tags.getter
    def tags(self):
        return [{'name': x} for x in self._tags.split("|")]
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s