The Future of TwUp

Regular readers of this website may note the difference between the style of this article and the others. We wanted to provide some insight into the future and possibilities for http://twup.me/ — the free audio sharing service targeted at visually impaired users of the Twitter micro-blogging service — and where better to do that than on the creator’s blog. We would like to apologize to those readers who have no connection to or interest in this audio-service, this is a one-time exception to the traditional contents of this site.

For the ones interested in TwUp.me, here is a brief rundown of the (potential) problems we are facing, plus the solutions we currently see before us.

The most obvious limitation, from the user’s point of view, is the available storage capacity. We have known about this since the project’s start in late June of this year. At the time of writing we are making ten gigabytes of space available to our users. This, again at the time of writing, guarantees that any given audio-file stays online for at least nine days before being deleted to make way for more recent additions to the service. The website offers a link to check the current “retention”. The fact that ten gigabytes of storage preserves a file for ten days after uploading leads to the second, less obvious, but even more important bottleneck: the amount of traffic consumed by TwUp is far greater than we had expected or planned for. This project started out as a private (or “elite”) service, as an answer to the alternatives of that time, and to utilize over-capacity of the server that BobbiLee.me also runs on. Since that time, TwUp was first integrated into the Twitter-Client The Qube, for those who were willing to compile The Qube’s source code themselves, and then this source code patch was provided in an easy-to-use setup program. Both these events have brought the service to a point where, as the retention indicates, nearly one gigabyte of audio is uploaded on a daily basis. Sometimes it’s more, sometimes it’s less. We expect this to relax over time, but it is still an impressive figure for a relatively small setup such as TwUp uses. Looking at the other side of the same coin, this audio gets shared and downloaded. Over the last two days this accounts for approximately twenty-five gigabytes of throughput, another impressive figure. While we are delighted and proud that our service has grown so popular and is so widely used after such a short time, it is taxing the limited monthly resources we can dedicate at this time. The limited storage capacity might be an annoyance, but the current traffic statistics for TwUp may well see it suspended to prevent going past the limit.

So what can we do about it? Quite a bit, ranging from a change to how the service is used, to plain and simple material support.

First and foremost, we would like to make absolutely clear that TwUp is not a free personal audio archiving service. Put more bluntly, we do not have the intention nor the desire to be a free webhosting service with virtually no limitations or conditions. The fact a file is likely to be deleted within two weeks of its being posted sounds harsh, but if one keeps the following two points in mind it turns out that it isn’t all that bad. First, TwUp was and is meant as “a quick audio posting service”. In this case, “quick” not only refers to ease of use, but also to the fact that we envision its postings to be just as much part of the present moment as are the tweets that they are linked in. For quick updates, quite possibly of little or no importance in a few weeks time, one uses Twitter. For longer, more elaborate updates, one usually sets up a weblog or similar “personal space”. We would like the same to be true for TwUp. For a quick capture of the moment it is perfectly usable, but to preserve valuable memories we strongly recommend a personal archive. We realize that the usage scenario we have in mind does not fully match what is currently going on. We don’t want to force people away and hope to meet everyone’s requirements to the best of our abilities. But from our perspective, storage capacity is not the most pressing point. This view is only strengthened by the second reason for not striving for increased retention. It is directly linked to the reasoning above: because of how quickly the social media flow it is not very likely that someone will go back to listen to a three-month-old audio posting. In fact, we are willing to assume that a ten-day-old posting will receive little or no traffic. The one danger here is that, as retention is limited, users will start to share even more “quick postings”, thereby further stressing the available bandwidth. But at present, it means that we are not primarily concerned about our storage capacity.

What we are concerned about, however, is the unexpectedly high traffic usage of the project. A lot of files can indeed safely be regarded as temporary in nature and can hence be deleted when the need arises. But temporary or not, they have been uploaded and then listened to, nibbling away at what bandwidth we have available. The options we may seriously consider are limited and straight-forward.

We could apply for more bandwidth, which is very likely to adequately deal with the problem. This is a relatively expensive option, though, and not one we will easily choose because it would involve considerable amounts of private money. While we value our users we are also very much aware that TwUp was intended for private to semi-public use and that personal investments can never be stopped once begun. Saying “no” isn’t pleasant, but first saying “yes” and then revoking what was promised is worse. We are already hovering dangerously close to this situation as it is.

A second, more future-proof solution is to contract a third-party hosting provider, one that offers plenty of storage space and bandwidth for a reasonable price (we are currently considering Servage as our provider). This too is out of the question if it has to be realized with private money, but is a lot less costly than the above alternative. Another advantage to this option is that we can offer more space for audio postings, something that extending our current server would not allow.

Finally, we have a third choice, but it is not one we are seriously considering because it would really not solve the problem but instead work around it for the time being. Several people have offered to host (part of) the TwUp audio archive, relieving the primary server somewhat. This plan has numerous shortcomings, some of which we will briefly outline here. Firstly, transferring audio from the primary server to the archive(s) means one extra download for every file uploaded; that is, one gigabyte a day at the current rate. Furthermore, this transfer would have to be properly coordinated; what if one of the servers is temporarily unavailable, for example. Then, the idea of having one or more third-party archives and presenting the owner(s) with little or no reward is not appealing. And lastly, as far as this discussion goes, spreading the project’s source code over multiple hosts that we do not own or control makes it more vulnerable and prone to inconsistency. We definitely value the support our users are giving us, but we have concluded that everything related to TwUp has to be active on only one logical host at any time.

These plans sound encouraging, but the question remains how we mean to realize them. Simply put, TwUp can evolve in one of three ways, from positive to rather negative. Each of these has its pros and cons, and we are still debating on them.

What we are hoping for is that we can raise the funds needed to safeguard TwUp for the next three years. We will not be making hasty decisions causing money to be wasted, especially if our users contributed said funds. The website, TwUp.me, has a button to donate (the donation amount has been preset to $10 USD for reasons of currency exchange rates). We would need at least twelve of such donations to cut clean without bringing private money into the picture. We are not selfish, only cautious. We — the owners — make little use of our own product, and without it we would have ample room for our files for many months to come. There is no logical explanation that justifies investing money so that other, more active users do not have to. This may sound harsh, but that does not change the situation. More involved readers will recall past experiences. On the other hand, we most certainly do not want those users who generously donate their much appreciated money to carry the weight of the “hitchhikers” on their shoulders. This is how the idea of offering higher quality services to donators was born. It is a tried and proven method. But, again, to do this we would need more resources. We are aware of several other initiatives to join in with TwUp, and so we don’t want to bind ourselves to a more spaceous TwUp unless its users genuinely wish for us to go that way. That means we will not have said resources available right now, and therefore cannot wholeheartedly and honestly promise them to prospect donators. Again, more involved readers will catch onto our fears. If everyone contributes a little, then noone loses a lot if something blows.

This leads to the second possible course for the project, one that is still not altogether negative. TwUp might just be able to sustain itself under the current conditions. This would become even more plausible if other services take over part of its users, maybe by offering the long-term storage and archiving that we have to reluctantly but firmly decline. The concept of TwUp lives and has potential, the danger involves leaving the service hanging between its current setup and a more professional (and expensive) system. Those who have such systems readily available might, instead of hosting some part of TwUp, start their own service. Our code is proprietary, but we hold no hostile feelings towards anyone. In fact, we more than welcome initiative! We don’t claim to have re-invented the wheel. We were just at the right place in a right time.

The third and final option, then, is simple and to-the-point: closing down. We do not fancy that.

We will utilize what donations we receive in the way we deem best for the continuation of the service, even if we can’t migrate to higher ground. We are currently in a test-run where we monitor the data usage as well as collecting some statistics — such as how often audio postings are accessed beyond their expiration date. We are also eager to hear from any current or future users, but especially from those who stopped using TwUp. We will be happy to improve the service, but we need to know where, what, and how.

There are only three rules to TwUp:

1. You are completely and exclusively responsible for the audio you upload into the public domain.

2. Donating is optional and will never become a requirement.

3. Baring point two in mind, you have no claim to our service, its contents and their future existence or lack thereof.

Thank You,

Bobbi Blood & Davy Kager