Introducing DTube: a decentralized video platform using STEEM and IPFS
I am really excited to finally bring this project to the public. Introducing...
DTUBE.VIDEO
What is DTube?
DTube is an application fully written in javascript, that runs in the
browser, that allows you to upload and watch videos on top of the IPFS
Network. Moreover, it uses STEEM as a database and enables earning
rewards from your uploads.
This might ring a bell for those who remember the SteemQ project
announcement, which made almost five thousand dollars in rewards, but
never got released and ended up being rebranded - It's still not
functional after more than a year, and even the current alpha uses a
back-end server for everything and is therefore still centralized. I am
sure I wasn't the only person disappointed by SteemQ.
I opted for a different approach. Build something first - talk after.
If you are wondering, I did everything by myself (and the help of open
source libraries of course) and it took about 4 months to reach what I
have now, starting from scratch.
A huge part of this article will be technical and maybe not so
interesting to some people, but I believe a big part of my followers are
interested in this kind of discussions, so I will explain how it works.
Sorry for the incoming walls of text :)
How to use DTube
Logging in
- Enter your STEEM username
- Enter your Private Posting Key, you can find it like so:
Multi-user login
Youtube and Facebook have this amazing feature that you can easily
have multiple channels linked to your account and switch between them
easily. I am sure I am not the only user on STEEM who owns multiple
accounts, and having to use different browsers or bots to upvote with
all accounts gets boring very quickly. On DTube, you can log in with as
many accounts as you want, and switch between them.
I don't know why SteemIt still doesn't have this feature.
Settings
DTube features a settings menu accessible from anywhere in the app.
It allows you to adjust your voting power, filter NSFW contents
similarly to SteemIt.com, and tune the IPFS Settings. IPFS Gateway is
where the files are loaded from, while the API is what we use for
uploading.
Uploading
First, you need to be logged-in to be able to access the upload page.
Then just simply add a video, a snap (aka thumbnail), and finally fill the texts for the blockchain and submit!
Then just simply add a video, a snap (aka thumbnail), and finally fill the texts for the blockchain and submit!
If all goes well, you should be taken to your video after submitting
and it should start playing the video between 0 and 30 seconds depending
on the gateway and your connection speed.
Browsing Videos
There are currently 3 main ways to discover contents on DTube. You
can go to the homepage and discover the NEW , HOT and TRENDING
categories. They work the same as SteemIt.
You can access a certain user's channel by going directly to https://dtube.video/c/curator for my testing account for example.
Finally, you can use the search in the top bar, more information on that later in the article.
You can access a certain user's channel by going directly to https://dtube.video/c/curator for my testing account for example.
Finally, you can use the search in the top bar, more information on that later in the article.
How DTube works
Using the STEEM Blockchain as a database
Because we want DTube to be truly decentralized, we
cannot have a server running a database and use it to query things.
Using a blockchain is a natural solution. STEEM has many advantages over
other blockchains. It is fast (3 sec blocks). It is free, anyone can
use it without having to deposit some form of currency, and transactions
have no fees. Who would want to pay money to upload a video, comment a
video, or even upvote? Nobody. As an added bonus, the STEEM blockchain
already has a reward-earning mechanism.
Any video uploaded on DTube becomes a STEEM content, that can earn rewards for 7 days. DTube doesn't use either the title or the body of a content to store information, therefore if you don't want your video upload to appear in your feed, you can edit or delete it, the video would stay intact on DTube. While it would be possible to display nothing on other platforms like steemit.com or busy.org, I believe the current behavior to be fine and will help the platform growth and the users rewards.
Any video uploaded on DTube becomes a STEEM content, that can earn rewards for 7 days. DTube doesn't use either the title or the body of a content to store information, therefore if you don't want your video upload to appear in your feed, you can edit or delete it, the video would stay intact on DTube. While it would be possible to display nothing on other platforms like steemit.com or busy.org, I believe the current behavior to be fine and will help the platform growth and the users rewards.
Using IPFS as a static file storage
IPFS is a protocol that enables decentralized file storage. The
principle behind it is called Distributed Hash Tables (DHT). The same
principle also powers the BitTorrent network. Just like how
cryptocurrency uses asymmetrical cryptography, DHT networks will hash
contents in order to be able to identify a file. The hash becomes the
identifier of a file, and it's as easy as re-hashing a file and making
sure the hashes match to ensure the file that was sent to us is the
original.
Why not use Torrent then? Torrent arguably already has more content and is more mainstream. However, Torrent was not built for the web. Some implementations such as WebTorrent are trying to make it work, but it still requires some 'hybrid' clients that would act as bridges in order to transfer files from hard clients such as μTorrent to web clients that run in a browser. Not very convenient, even less efficient. I am sure a lot of you have used PopcornTime to watch some 'pirated' content and have enjoyed it, but building something similar that runs inside a browser without having to run hybrid clients to connect people, is sadly impossible.
IPFS is a younger, open-source, and actively developed protocol. It has a bright future ahead. Using it as the main mean of storage for this project was a no-brainer.
Why not use Torrent then? Torrent arguably already has more content and is more mainstream. However, Torrent was not built for the web. Some implementations such as WebTorrent are trying to make it work, but it still requires some 'hybrid' clients that would act as bridges in order to transfer files from hard clients such as μTorrent to web clients that run in a browser. Not very convenient, even less efficient. I am sure a lot of you have used PopcornTime to watch some 'pirated' content and have enjoyed it, but building something similar that runs inside a browser without having to run hybrid clients to connect people, is sadly impossible.
IPFS is a younger, open-source, and actively developed protocol. It has a bright future ahead. Using it as the main mean of storage for this project was a no-brainer.
IPFS Storage Cost
IPFS is cool, but there is no magic. Someone needs to seed the files,
and your browser cannot permanently store huge files (local storage is
limited to 50MB on most browsers), so seeding through the app directly
is not possible as of today. While my first idea was to ask some
witnesses to run some IPFS nodes, it became clear after a few
conversations that this would create more problems than solutions as
most witnesses are non-technical persons and running and configuring an
IPFS node correctly seems to be a tough challenge for most. I still
believe this solution to be the right one, but I would clearly need to
setup a docker or something easy for witnesses to actively join the
DTube network and start seeding files (and earn a share of the rewards).
Instead, I searched for existing IPFS nodes and contacted the owner of IPFS Store,
a website that allows you to pay in Bitcoin to keep your files on the
IPFS network. After a few positive and instructional replies from @ridwant (steem, twitter/etc), I knew I found my man.
I have recently created the @dtube
account. This account will be used to collect 25% of the DTube author
rewards. 10% of these fees will be used to pay for long-term storage of
the files on IPFS Store.
The rate is $0.044 per GB per month. So, for example, let's say you
upload a 100MB video, that earns $10 rewards, then $0.25 will go to @ridwant
and ensure data redundancy for ~57 months. Once this time is passed,
users will need to either pay themselves (crypto accepted of course) to
keep the files being seeded, or seed it themselves directly on their own
PC and connection.
Design
Most developers know the saying 'do not reinvent the wheel'. The same
applies to design and user experience. The leading video platform
YouTube has been running successfully for years, and copying them makes
sense. Especially when I am a sucker at front-end development and UI
design. I tried to copy the look and feel of YouTube using the Semantic UI
framework. While I am proud of what it looks like, I am sure some more
talented people than me would slap me in the face for how bad it looks
and plays.
I am very open to suggestions. Or if you are a web-designer, with a
good portfolio, and Semantic UI experience, feel free to contact me on
the steemit.chat.
Using AskSteem for Search
It might be hard for non-technical people to understand it, but the
search is actually the toughest problem to make DTube a viable product.
When you have a standard SQL or Mongo database, searching is easy.
However here, our database is the STEEM blockchain, and we cannot search
by communicating with a steem node directly. I understand that search
is a low-priority feature for a blogging platform, especially when
Google indexes every content and offers a search API. But for DTube,
users will use it a lot.
I contacted @ridwant, a competent developer who posts on SteemIt but doesn't get half as much rewards as he deserves. He recently introduced AskSteem, a search engine for the STEEM blockchain. His service uses an ElasticSearch DB and will index new videos under 15 minutes.
His API was originally not enough for me to display certain
information such as the video snap (thumbnail) in the search results, or
else I would have needed to make one call for each search result, i.e. a
lot of useless calls. @thekyle fulfilled my request to improve his API in under 24 hours, and allowed it to run smoothly as it should.
Arguably, the search is not decentralized. Any search will go through the AskSteem server, search results could be altered, or the search could just stop working if @thekyle decides to close it.
A better solution would be to have a search function directly on the
steem nodes. Another solution would be to have yet-another DHT network
running inside the app, with a fuzzy search function. I have literally
no idea if making a fuzzy search algorithm on a DHT network is possible
without creating a point of failure, and I don't think anyone has an
answer to this question on Earth. That's why I decided to make a
compromise. We will see how it goes.
Permanently stored data
DTube will use your browser local storage to store a bit of
information. First, it will store your user accounts and posting keys so
that you don't have to relog every time. I suppose that won't shock
anyone considering steemit.com does the same.
Secondly, DTube also stores your watched videos in order to display
the 'Watch Again' category. Just like YouTube does it. You can notice
that post rewards won't show for the Watch Again category, as these
rewards can change at any point in time and it would be impossible to
trust the data we stored a while back ago.
If you clean your browser sometimes, you will have to relog and your
watched videos history will be cleaned, with no way to get them back.
Just a friendly warning :)
Why can't I log in with my master password or active key?
Because it's a retarded thing to do. Steemit.com allows it solely
because they also have a wallet in the app. XSS attacks are common on
the internet, and most devs are rightfully paranoid about it. I would
feel terrible if some users of my app lost their accounts and money
because of me. Instead, it is much easier to force users to use the
posting key and limit the risks.
DTube does not include any feature that requires anything else than
the posting key. That means no wallet, no password reset, no internal
market trading, no powering down, and way less work for me. If you want
to manage your STEEM rewards, I invite you to use Vessel. And only use your posting key on any online platform such as steemit.com, busy.org, etc.
"What about SteemConnect??" I can see this as a comment below this
article already. I am very aware of the SteemConnect project, that aims
to provide an easy way for developers to enable STEEM login and key
management. However, I am personally very much against delegating my
keys or sending them to someone else to use, even if it's a trusted 3rd
party ran officially by SteemIt Corp. Sorry but I like to remain in 100% control of my keys, and you should too. Remember all the people who cried on Reddit whenever an exchange got hacked? Don't be like them.
The case for user-side encoding
On YouTube, you upload your original file, and YouTube will encode
your file and even convert it to multiple qualities, as well as create
some fancy gif to preview the video on mouse hovering.
Achieving the same in DTube is hard. Encoding a video file in javascript is harder to code, and would always be slower. Bad
On top of that, if you want multiple quality files, then you would need to upload multiple files and multiply the time it takes to upload your video.
Bad Bad
Finally, it would probably grow the total size of the DTube application quite a bit, and therefore increase the initial loading time. Bad Bad Bad
On top of that, if you want multiple quality files, then you would need to upload multiple files and multiply the time it takes to upload your video.
Bad Bad
Finally, it would probably grow the total size of the DTube application quite a bit, and therefore increase the initial loading time. Bad Bad Bad
There are already countless programs that permit
video encoding. And that would only be required to do for videos
recorded with professional tools such as high-grade cameras. People who
own this kind of hardware, usually already know about encoding. My
personal smartphone already encodes every video in H.264, and I suppose
your iPhone does the same. So why not keep it stupidly simple?
Easy to integrate for other platforms
Any other STEEM based platform can easily integrate videos posted on DTube. The video IPFS hash is stored inside the
json_metadata of a post. Then it is as easy as doing Known Issues
- No progress bar on the upload - github issue
- Avatars don't resize and load the full-size images - maybe solution
Missing features
- Reputation system
- Tags browsing
- Video suggestions (on the right of the video)
- Following Channels
- Custom Player
I believe the current state of the product to be viable, even without
these missing features. I will add them, slowly. Also because I am
unsure which of those is a priority right now, I will let you guys
choose and argue about it in the comments!
App Dependencies
DTube is built with Meteor, and uses many open-source packages from NPM. DTube wouldn't exist without them, so they deserve some credits.
ipfs-api - Communication with the IPFS Network
SteemJS - Communicating with the STEEM Blockchain
autolinker - Adding links to descriptions / comments
moment - Displaying times
XSS - For displaying untrusted texts and preventing XSS attacks
Credits to debuggers and idea suggesters
@steeminator3000 @dailydogger @coldbolt @nannal @thekyle @ekitcho @fabien @kalvas @french.fyde @nnnhhh @fayetristan @lukestokes
@ridwan
Cross-links
I'm getting literally 0 traction for this article on other mainstream
social networks despite the correlation with IPFS. Spare an upvote?
Let's try to bring attention to the outside world.
Reddit
Hacker News
Last words
While I personally believe that decentralization is the future, don't
get your hopes too high yet. Competing with YouTube is an heavy task
that cannot be done by one man. My intent with this project was to
practically demonstrate the possibility of decentralized video on top of
IPFS. It works, but we don't know how well it will work in the
long-term. YouTube also has a lot of extra features that makes the
experience superior. What I built, represents maybe 1% of the work
required to have something that can compete with YouTube. I built some
walls, but there is no insulation, or painting, and the whole building
might collapse at any time.
by: @ridwant
source:
إرسال تعليق