Revealed: The Secrets of YIFY-Torrents’ Network

[Ernesto]

Needing little introduction, YIFY (now correctly called YTS) is one of the world’s most popular sharing sites. At the start of 2014 it was the 5th most popular torrent site in the world and according to brand new figures provided to us yesterday, the site is currently handling around 10 million visits per month.

Speaking with TF, site developer Jduncanator says that since last month’s rebranding the team have been not only working on new features, but also making sure that existing processes are fine-tuned. The dev says that user experience at YTS is paramount.

Don’t make users wait

“Studies show that 100ms is how long you have for the user to feel like the task was instantaneous. One second is how long you have for the user’s state of flow to remain uninterrupted (though the delay will still be noticeable). Ten seconds is how long you have before the user loses interest entirely and will want to multi-task while the task is completing,” JD explains.

“That 100-1000ms sweet spot is something most web developers aim for, but whats even more surprising is that only around 5% will actually achieve that. An average visitor will spend between 30 seconds to 4 minutes on YTS, so each individual page load will make up a large proportion of their overall user experience. This is what we wanted to focus on heading into 2014.”

Google-powered

One key upgrade towards achieving that aim is the inclusion of a brand new protocol called Spdy (pronounced Spee-dy), a Google-developed protocol designed to shift web content.

“Google’s new protocol enables extremely low latency round-trip times and compression of headers. All leading (up-to date) browsers support SPDY 3 and so now do our servers allowing for high speed access and connection to all of our site,” JD reveals.

While most site visitors won’t have any idea that things have changed, JD says that behind the scenes that’s definitely not the case. Network infrastructure has undergone significant upgrades, enhancing not only speed but security and overall experience.

Security

“With the latest NSA revelations and with overall ‘security’ paranoia going through the roof we decided to revise our current security setup to see if we could do anything better – turns out we could,” JD says.

“We used to have 2048-bit RSA keys. These by themselves don’t pose a security risk, but if we are to go by anything the NSA reports show, they could possibly have the technology to crack these more efficiently than the public. To be on the safe side, we decided to double the key size to 4096-bit RSA which effectively multiplied the amount of time to crack our encryption keys by 2³².”

And then, despite our requests to keep things pretty simple, JD just couldn’t contain himself, detailing the site’s BEAST attack protection, OCSP stapling enabling and HSTS implementation. Needless to say, he’s passionate about the site and keeping it finely tuned – fortunately he doesn’t mind if those listening need to Google acronyms every two minutes either.

“You can see a report of our SSL over here. We score an A+ and are in the top 3%, something even Google couldn’t achieve,” he proudly adds.



Network setup – On the Edge

YTS operates two levels of network – Edge and Internal. The Edge network is an array of servers running the open source NGINX high-performance HTTP server and reverse proxy software. These servers are what users connect to when visiting the site. The Edge servers communicate with the servers on the Internal network, keeping the latter hidden from the public eye.

“The idea of these reverse proxies is that they simply forward web requests to our ‘hidden’ network and then pass the response back to you. This means that all you ever see is our cheap web servers’ IP addresses and never our internal networks. It allows us to have large amounts of high capacity connections to gobble up any ‘naughty’ or attack packets trying to take down our site and allows us to get our static content (images) closer to our end users, improving page load time and overall User experience.”

YTS’ ‘Edge’ network proxies, locations marked with blue squares

“Because the closer you are to the server, the faster the connection, we obviously want as many servers as possible as close to as many countries as possible. In the last two days we have added 13 new edge servers all around the world. Up until then, we only had 9 edge servers located around Europe but we wanted to aim big and get our content closer to our users to obtain faster load times. We now have 22 Edge servers.”

JD says that initially the YTS setup was modeled on Cloudflare’s, but now the site has servers in more locations than Cloudflare does.

“With these now in place and serving content, you should immediately see an improvement in screenshot and static content load time as well as faster overall page load speeds if the Edge node is a reverse proxy,” he notes.

Network setup – On the inside

“Our Internal network is made up of a few dedicated servers that are located in a hidden location far away and never see a public internet request EVER,” JD explains. “There are many reasons behind this from mitigating DMCA issues and annoyances to DDoS and attack protection.”

YTS runs two other main servers with 16 cores each, easily allowing them to cope with the traffic generated by the site’s users. All key infrastructure is in one location allowing for easier maintenance and backups. Perhaps understandably JD didn’t share any additional information on the Internal network and there is certainly no map indicating server location.

However, the diagram below shows the YTS setup, with the user at the bottom communicating with the site’s Edge servers which in turn talk to the Internal setup consisting of a main web server and dedicated database server.



What is evident when speaking with JD is that the site’s operators are not only passionate, but also blessed with seemingly endless energy and enthusiasm. The upgrades detailed above are just the beginning too, with yet more tweaks to come.

“Over the next few weeks we will be playing with our DNS settings to try and even out our load amongst servers and to make pings even LOWER. Updates to our DNS infrastructure now mean that all users will be given IPs for Edge servers that are closest to them and should provide the lowest pings possible.

“We’ll always be looking for ways to improve user experience and we plan on optimizing site code over the next few months to try and obtain even more speed out of the site,” JD concludes.

Source: TorrentFreak, for the latest info on copyright, file-sharing and VPN services.

source

[Picklock]

umm...now how our hamster can compete with this?
uh I forgot...our hamster has many balls of steels