Image for post
Image for post

Photo by Chris Ried on Unsplash

We at the Open Library of Humanities and Birkbeck CTP have been working on a Repository system initially built to provide a space for Preprints. We came across an issue where once live data had been imported into the system the main dashboard began to slow down. This wasn’t unexpected — when you add a tonne of data sometimes things get a bit slower — but this was excruciating.

A little investigation with the Django Debug Toolbar showed that we were repeating a bunch of queries over and over again. This is quite easy to tidy up — we added some select_related() and prefetch_related() calls to limit the number of additional queries being made in the templates but due to the nature of one of our models the page remained relatively slow to load. …


The Open Library of Humanities main site, and Orbit and IJWWE journal sites are all on Janeway and are hosted on Reclaim Hosting’s great Shared Hosting service. Reclaim leverage Digital Ocean’s VPS and Block Storage services to provide this service, so when DO’s Frankfurt data center fell to pieces on the 1st of April (Apr 01, 2018–07:31 UTC) we were among the unfortunate victims.

Our sites were offline from 0731 on 01/04 to 1937 02/04. This length of downtime, we feel, isn’t okay so we are setting out to ensure it doesn’t happen again. We plan to:

  1. Setup a second Reclaim account in a different data center/region…

Recently I wanted to add a UUID field to an existing model called Institution, this model represents the Institutions that support the Open Library Humanities and the UUID field would represent a referral code. I added the following line to models.py and ran makemigrations.

referral_code = models.UUIDField(default=uuid.uuid4)

Unfortunately, this works perfectly for new instances of Institution, it will generate the same UUID for all of the existing objects. To get around this we need to alter the migration that has been generated.

# -*- coding: utf-8 -*-
# Generated by Django 1.11.5 on 2018-02-26 19:29
from __future__ import unicode_literals

from django.db import migrations, models
import uuid


def create_uuid(apps, schema_editor):
Institution = apps.get_model('consortial_billing', 'Institution')
for inst in Institution.objects.all():
inst.referral_code = uuid.uuid4()
inst.save()


class Migration(migrations.Migration):

dependencies = [
('consortial_billing', '0031_supportlevel_description'),
]

operations = [
migrations.AddField(
model_name='institution',
name='referral_code',
field=models.UUIDField(blank=True, null=True),
),
migrations.RunPython(create_uuid),
migrations.AlterField(
model_name='institution',
name='referral_code',
field=models.UUIDField(default=uuid.uuid4, …


Allowing people to upload files to your server can be fraught with peril, we don’t want people to upload malicious files or binaries. In some cases we want to limit uploads to very specific file types, PDF or DOCX for example.

Python has a mimetype package that can be used to check the mime of a file lets say we upload a new jpeg file:

import mimetypesfile_mime = mimetypes.guess_type('a_file.jpg')>> ('image/jpeg', None)

This is great! Python has guessed that the file is a JPEG. But what if we upload a file that is actually a pdf:

import mimetypesfile_mime =…


Janeway (an open source publishing platform written in Python and using the Django framework) uses an event system that allows apps and plugins to register to be called when an event is fired. An example of this would be when a new submission is made.

event_logic.Events.ON_ARTICLE_SUBMITTED

This event is called whenever a new article is submitted, any app or plugin can register to have a function called when this event is triggered by the following method:

from events import logic as event_logicevent_logic.Events.register_for_event(event_logic.Events.ON_ARTICLE_SUBMITTED, plugins.my_submission_plugin.events.a_function)

The above is obviously an example, but you get the picture. By default an Event passes a set of keyword arguments for your function to use and these are documented inside events.logic.Events


Monkey-patching software is generally frowned upon, but there is a time and a place for everything, even monkey-patching.

Take Janeway for example, we were working nicely until we had a feature request from a developer who was thinking about using it: Could we handle multi-tenant journals without using multiple domains? It made sense, not every organisation who would be interested in using this software would have the in house capacity to manage domains and subdomains when they have tens of journals so we worked out how it could be done.

Currently we were using subdomains like:
ctp.localhost and we needed to move to a path based identifier eg. localhost/ctp/ so instead of using the domain itself as the identifier we needed to use the first path element, this required an updated of our…


What does this mean!? Well, lets make an example.

class Company(models.Model):
name = models.CharField(max_length=100)
logo = models.ForeignKey(File, null=True, blank=True)
class File(models.Model):
filename = models.CharField(max_length=100)
mime = models.CharField(max_length=100)

Say we have an Company object and a File object. The file object is set to be the logo.

company = models.Company.objects.create(name='A Company')
file = models.File.objects.create(filename='file.jpg', mime='jpeg')
company.logo = file
company.save()

If we then delete the file

file.delete()

Then we will actually also delete the company object because by default django will, on delete, cascade. So, we have to set the on_delete attribute to avoid this.

class Company(models.Model)
name = models.CharField(max_length=100)
logo = models.ForeignKey(File, null=True, blank=True, on_delete=models.SET_NULL)

If the File object is deleted nowm the logo will be set to NULL rather than deleting the object.


I’m currently converting chunks of code that make data changes on GET requests to use POST within the Django environment and thought it might be useful to document how I did it here.

Here is a snippet of code using a GET request to change data using a query string to pass the id to be deleted.

# Link in the Template
<a href={% url 'a_url' %}?delete={{ item.pk }}>Delete Me!</a>
# Code in View
if 'delete' in request.GET:
item_pk = request.GET.get('delete')
item = get_object_or_404(models.Item, pk=item_pk)
item.delete()
return redirect(reverse('a_url'))

Converting this to use POST isn’t terribly difficult.

# We've got a button in a form now
<form method="POST">
{% csrf_token %}
<button type="submit" name="delete" value="delete">Delete Me!</button> …

On the 12th of October I will be moving on from my post as Technical Architect at Ubiquity Press. I’ve had a great time working on cutting edge publishing technology and working alongside the teams at UP and our partner presses to provide a great experience

During my tenure with Ubiquity we’ve developed a fair few Open Source projects (Rua, OJSSQLA, Toru) that can be found in the Ubiquity Press Github account and brought 12 new presses/partners on boar including:

Including our partners we are now hosting 170 full journals as well as providing hosting services to 6 more…


Image for post
Image for post

At Ubiquity Press our new Junior Developer, Stuart Jennings is working to integrate Fundref with our submission platform, allowing authors to submit their Funding info so it can be published and deposited with Crossref.

We were informed of the Fundref Widget (info, github), which we used as the basis of our integration, with a slight modifications so that we can get hold of the Funder ID as well as their official name (required for deposit) and to bind the funder names to the award numbers.

This all went fine until Stuart started noticing blank entries in our POST dictionary:

<QueryDict: {u'abstract': [u''], u'title': [u'd'], u'csrfmiddlewaretoken': [u'iO4X7u8TAS6Ew6sTGu9mAEOr1QyH7d47'], u'has-funding': [u'True'], u'has-competing': [u'False'], u'funder-name-2': [u'Cambridge University Hospitals'], u'funder-name-3': [u'Fake Uni'], u'funder-name-0': [u''], u'funder-name-1': [u'Oxford University Hospitals NHS Trust'], u'section_id': [u'5'], u'funder-name-4': [u''], u'funder-name-5': [u''], u'award-number-3': [u'', u''], u'award-number-4': [u'', u''], u'award-number-5': [u'', u'1213'], u'award-number-2': [u'', u'Cam1', u'Cam2', u''], u'award-number-0': [u''], u'award-number-1': [u'', u'Ox1', u'Ox2', u'Ox3']}>…

About

Andy Byers

Pub Tech developer for Birkbeck CTP

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store