[EN] The future of eRepublik-Deutschland.de
Day 3,378, 03:08
•
Published in Germany 
•
by
•
by Master_rg
Hello fellow citizen,
In this article I would like to give you some updates about the development of eRepublik-Deutschland.de and its successor eRepublik.tools. In order to make everything a bit more clear, let me start with a small table of contents.
Table of contents
1. I don’t like to read
2. What is eRepublik.tools
3. Why do you build a successor instead of improving eRepublik-Deutschland.de
4. What are the plans of eRepublik.tools
5. What is its current state
6. When are you going to release it
1. I don’t like to read
Everything becomes better, eRepublik.tools will replace eRepublik-Deutschland.de in future, possible release as mobile app, no ETA yet, still a lot of stuff to do, I still play eRepublik, eRepublik-Deutschland.de won’t be shut down in the next months, #MakeLuvNotCoup, if you got too much money I'll take it.
2. What is eRepublik.tools
As I said it already, eRepublik.tools is the successor of eRepublik-Deutschland.de, a kind of highly optimized version of the current tool collection.
3. Why do you build a successor instead of improving eRepublik-Deutschland.de
eRepublik-Deutschland.de was planned as a small german tool website in the past. Within months we became the biggest tool website in eRepublik. In general there are two reasons for a change: First of all, people still have issues with such a long domain name (some search words excerpts from google: http://prntscr.com/ea7rgo, http://prntscr.com/ea7rnb) which is of course funny in a certain way BUT of course not the best way to handle such a situation.
On the other hand I am not that satisfied with the code quality. Of course everything is running fine and without any issues, but in certain situations I can see that some decisions in design weren’t the best ones I would need right now. This “issue” is caused by the fact that eRepublik evolves, which is, of course, not a bad thing. However: Things that were a good design decision more than a year ago don’t need to be a good decision in the current situation. Of course I could try to adapt the changes at eRepublik-Deutschland.de but due to the deep connection of literally all modules this would introduce regular downtimes which include more than 70% of the website - and that could be quite stressful in certain situations.
So instead of performing an open heart surgery which would introduce bigger downtimes at the current page, I am completely starting from scratch taking time to implement everything mostly future-proof. Thanks to eRepublik-Deutschland.de I am not in a hurry as we got working and stable tools which support your daily life in eRepublik.
Last but not least I am also writing a research paper in optimization strategies for my university - so it’s quite handy to test and use it in practice.
4. What are the plans of eRepublik.tools
The plans are mostly the same compared to my previous posts about eRepublik-Deutschland.de. I would like to optimize the overall performance, so I am not only talking about the backend (including webservers, databases, caches) but also the frontend (the stuff you see when you open a website).
In detail we are moving to a PHP framework called Symfony on the backend (@Dani if you read this: Please take a selfie right now), and on the frontend we are moving to Angular 2. The benefit is quite huge: On the one hand I am able to create a modern backend and on the other hand Angular 2 makes it possible to create a so called “Single Page Application”.
In general you can compare a Single Page Application with an app on your mobile or tablet, but instead of installing it, it directly runs in your browser. Furthermore it’s also possible to wrap this application up and release it as an app on app stores for Android, iOS and Windows. This should take eRepublik tools to a new next level.
To give you a better insight about our planned backend, I’ve got two little images to show you the current and the planned situation (attention: those images are just sketches, some things are way more complicated than shown in the images):
Current Situation: http://prntscr.com/ea8sno
Planned Situation: http://prntscr.com/ea8saj
If you take a look at the current situation, we are using MariaDB as a priority #1 data source for our backend. Basically MariaDB is a huge database server which has all the data. Memcached on the other hand is a so called in-memory database system. Instead of writing stuff on your hard drive, it just saves everything right in your RAM which is super fast. But there’s one problem: We can’t cache all our objects in Memcached due to some design decisions / bugs that are caused by object serialization - and that’s the reason MariaDB is visualized as Priority #1 in the database part.
You can also see that there isn’t a real dynamic part in this chart: If a user or a service wants new and fresh data, it needs to query the Public API in order to receive updates which results in clients hammering on the server while waiting for new data.
The planned situation is slightly different: First of all we did some changes to the database and cache: Thanks to our new design we are able to save our new objects right in our new in-memory database Redis. Instead of querying MariaDB most of the time, we are now able to call Redis which is just super fast. Also, when we are updating our data (like statistics and stuff) we are writing it to both systems: MariaDB and Redis, but as soon as an application needs the data, Redis is called first, reducing the overall load of our database server.
The next huge difference is the so called Change- / Event- Detection Strategy. In the current implementation I had to check every field more or less manually if it has changed and then e.g. save the change to the database (nick history for example). In the new system this happens completely automatically, I just tell the system “Hey, watch if the nick of the citizen changes! If it changes: Save it” - and the system automatically takes care of it. This system can be used on all different aspects of the game: Listen to changes in the ministries of a country? No problem. Listen to a change of the current BH/CH in a battle? Nothing’s easier than that.
This part becomes even more interesting if you take a look at the WebSocket. So, what’s a WebSocket? In general it is a way to communicate between your browser and my server in realtime. In combination with the change detection strategy it becomes a powerful feature as I can tell your browser in real time when e.g. someone tries to steal your BH medal or you’ve been overtaken during the presidential election. No need to refresh the page, you’ll get a notification just in the second we detect a change.
I think you can see that this will push the user experience to a completely new level, especially if you think of those realtime notifications right on your mobile.
But of course, some changes aren’t visible for everyone: I’ve also created a new crawling and monitoring system. In detail this means that our crawlers (those things that visit eRepublik.com) use less resources on our server, but are also less stressful to the servers of eRepublik - while offering a bigger amount of features.
In the new system I get notifications completely automatically, so in case eRepublik changes the source code and my crawlers stop working, I will receive a message in real time. Furthermore I can check all modules manually on the server console or directly in the browser. You can see a small example right here: http://prntscr.com/ea92tr
Other optimizations are performed on the deeper core of the system. Thanks to the switch to PHP 7 and some adjustments to OPcache, scripts get loaded faster than ever before. So, to explain it: If you visit the website and check your citizen statistics, the script which hands out those information needs to be parsed and compiled before. After that it gets executed and sends your statistics - and this happens every time you request the data.
The new system is slightly different: The specific script gets parsed and compiled only once, afterwards it’s cached until I change the source code. I think you can see the benefit without being a programmer.
5. What is its current state
Basic planning started some weeks ago, the last week was about creating a first basic backend including a new crawler engine, notification system, change / event detection strategies. In order to give you an overview of stuff that’s already implemented and stuff that still needs to be done, let’s take a look at a checklist:
Crawler
[x] New Crawler Engine
[x] eRepublik API support
[x] Citizen Crawler
[x] Country Crawler
[x] Region Crawler
[o] Organization Crawler
[o] Alliance Crawler
[o] Newspaper Crawler
[o] Battle Crawler
[o] Leaderboard Crawler
[o] Market Crawler
Statistics
[x] Country Statistics
[x] Region Statistics
[o] Citizen Statistics
[o] Organization Statistics
[o] Alliance Statistics
[o] Newspaper Statistics
[o] Battle Statistics
[o] Damage Statistics
[o] Market Statistics
Misc
[o] Registration / Login System
[o] WebSocket implementation
[o] Notification System
[o] New API for developers
This is just about the backend, so the stuff my server needs to deal with in order to present some data. As soon as most of those things are finished, I am going to start to work on the frontend (this Angular 2 stuff in the beginning, do you remember it?) - but that’s going to be a bit faster than implementing the backend.
But, again: In order to be able to quickly implement the frontend part, it needs quite some time and planning on the backend side. There’s still a lot of stuff to do, but I am sure we will see quite some progress in the next weeks. In case you are interested in some testing in future, don’t hesitate to send me a message - I’ll then add you onto my list of beta testers.
6. When are you going to release it
When it’s done. I can’t tell you an ETA but I hope to be able to start a first closed beta in the next 3 to 4 weeks. It all depends on other stuff I have to deal with besides eRepublik - because somehow the server needs to be financed
Oh and btw: In case you ask yourself what’s going on with the “make eRepublik great again platform” I was talking about in the last articles: It’s going to be implemented right into the new tools page as its own category
So in case there are more questions, ideas, feedback: Don’t hesitate to leave a comment, I would enjoy some discussions with you.
In this article I would like to give you some updates about the development of eRepublik-Deutschland.de and its successor eRepublik.tools. In order to make everything a bit more clear, let me start with a small table of contents.
Table of contents
1. I don’t like to read
2. What is eRepublik.tools
3. Why do you build a successor instead of improving eRepublik-Deutschland.de
4. What are the plans of eRepublik.tools
5. What is its current state
6. When are you going to release it
1. I don’t like to read
Everything becomes better, eRepublik.tools will replace eRepublik-Deutschland.de in future, possible release as mobile app, no ETA yet, still a lot of stuff to do, I still play eRepublik, eRepublik-Deutschland.de won’t be shut down in the next months, #MakeLuvNotCoup, if you got too much money I'll take it.
2. What is eRepublik.tools
As I said it already, eRepublik.tools is the successor of eRepublik-Deutschland.de, a kind of highly optimized version of the current tool collection.
(Official photo with Plato during this announcement)
(Seems like Plato forgot his gold jewelry - or did he sell it already?)
3. Why do you build a successor instead of improving eRepublik-Deutschland.de
eRepublik-Deutschland.de was planned as a small german tool website in the past. Within months we became the biggest tool website in eRepublik. In general there are two reasons for a change: First of all, people still have issues with such a long domain name (some search words excerpts from google: http://prntscr.com/ea7rgo, http://prntscr.com/ea7rnb) which is of course funny in a certain way BUT of course not the best way to handle such a situation.
On the other hand I am not that satisfied with the code quality. Of course everything is running fine and without any issues, but in certain situations I can see that some decisions in design weren’t the best ones I would need right now. This “issue” is caused by the fact that eRepublik evolves, which is, of course, not a bad thing. However: Things that were a good design decision more than a year ago don’t need to be a good decision in the current situation. Of course I could try to adapt the changes at eRepublik-Deutschland.de but due to the deep connection of literally all modules this would introduce regular downtimes which include more than 70% of the website - and that could be quite stressful in certain situations.
So instead of performing an open heart surgery which would introduce bigger downtimes at the current page, I am completely starting from scratch taking time to implement everything mostly future-proof. Thanks to eRepublik-Deutschland.de I am not in a hurry as we got working and stable tools which support your daily life in eRepublik.
Last but not least I am also writing a research paper in optimization strategies for my university - so it’s quite handy to test and use it in practice.
🙂
4. What are the plans of eRepublik.tools
The plans are mostly the same compared to my previous posts about eRepublik-Deutschland.de. I would like to optimize the overall performance, so I am not only talking about the backend (including webservers, databases, caches) but also the frontend (the stuff you see when you open a website).
In detail we are moving to a PHP framework called Symfony on the backend (@Dani if you read this: Please take a selfie right now), and on the frontend we are moving to Angular 2. The benefit is quite huge: On the one hand I am able to create a modern backend and on the other hand Angular 2 makes it possible to create a so called “Single Page Application”.
In general you can compare a Single Page Application with an app on your mobile or tablet, but instead of installing it, it directly runs in your browser. Furthermore it’s also possible to wrap this application up and release it as an app on app stores for Android, iOS and Windows. This should take eRepublik tools to a new next level.
To give you a better insight about our planned backend, I’ve got two little images to show you the current and the planned situation (attention: those images are just sketches, some things are way more complicated than shown in the images):
Current Situation: http://prntscr.com/ea8sno
Planned Situation: http://prntscr.com/ea8saj
If you take a look at the current situation, we are using MariaDB as a priority #1 data source for our backend. Basically MariaDB is a huge database server which has all the data. Memcached on the other hand is a so called in-memory database system. Instead of writing stuff on your hard drive, it just saves everything right in your RAM which is super fast. But there’s one problem: We can’t cache all our objects in Memcached due to some design decisions / bugs that are caused by object serialization - and that’s the reason MariaDB is visualized as Priority #1 in the database part.
You can also see that there isn’t a real dynamic part in this chart: If a user or a service wants new and fresh data, it needs to query the Public API in order to receive updates which results in clients hammering on the server while waiting for new data.
The planned situation is slightly different: First of all we did some changes to the database and cache: Thanks to our new design we are able to save our new objects right in our new in-memory database Redis. Instead of querying MariaDB most of the time, we are now able to call Redis which is just super fast. Also, when we are updating our data (like statistics and stuff) we are writing it to both systems: MariaDB and Redis, but as soon as an application needs the data, Redis is called first, reducing the overall load of our database server.
The next huge difference is the so called Change- / Event- Detection Strategy. In the current implementation I had to check every field more or less manually if it has changed and then e.g. save the change to the database (nick history for example). In the new system this happens completely automatically, I just tell the system “Hey, watch if the nick of the citizen changes! If it changes: Save it” - and the system automatically takes care of it. This system can be used on all different aspects of the game: Listen to changes in the ministries of a country? No problem. Listen to a change of the current BH/CH in a battle? Nothing’s easier than that.
This part becomes even more interesting if you take a look at the WebSocket. So, what’s a WebSocket? In general it is a way to communicate between your browser and my server in realtime. In combination with the change detection strategy it becomes a powerful feature as I can tell your browser in real time when e.g. someone tries to steal your BH medal or you’ve been overtaken during the presidential election. No need to refresh the page, you’ll get a notification just in the second we detect a change.
I think you can see that this will push the user experience to a completely new level, especially if you think of those realtime notifications right on your mobile.
But of course, some changes aren’t visible for everyone: I’ve also created a new crawling and monitoring system. In detail this means that our crawlers (those things that visit eRepublik.com) use less resources on our server, but are also less stressful to the servers of eRepublik - while offering a bigger amount of features.
In the new system I get notifications completely automatically, so in case eRepublik changes the source code and my crawlers stop working, I will receive a message in real time. Furthermore I can check all modules manually on the server console or directly in the browser. You can see a small example right here: http://prntscr.com/ea92tr
Other optimizations are performed on the deeper core of the system. Thanks to the switch to PHP 7 and some adjustments to OPcache, scripts get loaded faster than ever before. So, to explain it: If you visit the website and check your citizen statistics, the script which hands out those information needs to be parsed and compiled before. After that it gets executed and sends your statistics - and this happens every time you request the data.
The new system is slightly different: The specific script gets parsed and compiled only once, afterwards it’s cached until I change the source code. I think you can see the benefit without being a programmer.
5. What is its current state
Basic planning started some weeks ago, the last week was about creating a first basic backend including a new crawler engine, notification system, change / event detection strategies. In order to give you an overview of stuff that’s already implemented and stuff that still needs to be done, let’s take a look at a checklist:
Crawler
[x] New Crawler Engine
[x] eRepublik API support
[x] Citizen Crawler
[x] Country Crawler
[x] Region Crawler
[o] Organization Crawler
[o] Alliance Crawler
[o] Newspaper Crawler
[o] Battle Crawler
[o] Leaderboard Crawler
[o] Market Crawler
Statistics
[x] Country Statistics
[x] Region Statistics
[o] Citizen Statistics
[o] Organization Statistics
[o] Alliance Statistics
[o] Newspaper Statistics
[o] Battle Statistics
[o] Damage Statistics
[o] Market Statistics
Misc
[o] Registration / Login System
[o] WebSocket implementation
[o] Notification System
[o] New API for developers
This is just about the backend, so the stuff my server needs to deal with in order to present some data. As soon as most of those things are finished, I am going to start to work on the frontend (this Angular 2 stuff in the beginning, do you remember it?) - but that’s going to be a bit faster than implementing the backend.
But, again: In order to be able to quickly implement the frontend part, it needs quite some time and planning on the backend side. There’s still a lot of stuff to do, but I am sure we will see quite some progress in the next weeks. In case you are interested in some testing in future, don’t hesitate to send me a message - I’ll then add you onto my list of beta testers.
6. When are you going to release it
When it’s done. I can’t tell you an ETA but I hope to be able to start a first closed beta in the next 3 to 4 weeks. It all depends on other stuff I have to deal with besides eRepublik - because somehow the server needs to be financed
😉
Oh and btw: In case you ask yourself what’s going on with the “make eRepublik great again platform” I was talking about in the last articles: It’s going to be implemented right into the new tools page as its own category
🙂
So in case there are more questions, ideas, feedback: Don’t hesitate to leave a comment, I would enjoy some discussions with you.
Facebook
Twitter
Discord
Comments
v
o7
v
at least you have something to do :3
V
v/s/endorse, can you make article for this month about buying packs ??
o7
Please take a closer look at the official photo together with Plato. An updated version will be released the day after the pack-selling has finished.
Why haven't they hired you yet? You'd complete 5 years worth of their work in just 1.
+1 for Symfony
Don't fetch..push.
Sounds like a very interesting project.
I have not understood most of what u said, but if u say it will be better, let's believe you.
You are doing a great job for eRepublik \o/
3 -4 years too late....
-1 for Smyfony 🙂
If you will use a clientside ( bloatware 😛 ) framework like Angular , I don't think you need server side rendering thanks to nature of this project. So a simple backend would suffice imo.
Well, "simple" is always relative. The current backend is quite simple but it doesn't give me the needed amount of "freedom and flexibility" I need in the long run.
I don't have the time to e.g. check logs for issues or test everything on my own, thanks to Symfony and the current implementation it's relatively easy for me to keep track of everything.
Furthermore Symfony forces you to use higher standards of software engineering which leads to more sustainable code (at least it should). In the backend I was always like "yeah well, it works, it's not the best solution but it's fine" - and right now I reached a point where all this is, of course, still managable but it's not "fun" anymore.
Also, seeing the serverload caused by the issues with memcached, isn't that funny from time to time. Again: It's all within the limits but I hate to see that kind of stuff which isn't even close to "being perfect".
Tbh, never liked frameworks as long as they're not sugarcoat thin. Just wondered a simple REST would work as long as most of job is handled by Angular.
Still eager to see where it leads as I might inspire for my project 🙂
Yeah but it's not only about serving the data, it's also about gathering everything while also finding a balance between reliability and performance.
Right now in the new system I am using mapped objects in most cases and good old prepared statements for performance critical database jobs which provides the best balance in my case.
Of course you could just do everything in a functional programming style, but at a certain point it'll become a pain in the ass - especially if you don't work with your stuff everyday.
Symfony itself forces you do use best practices / PSR's in most situations. If every PSR also is the best practical solution? Definitively not (at least for smaller projects), however, does it increase the sustainability of your code: It does. And that's the point as I don't want to "re-read" hundred lines of code just to understand what I did over a year ago.
Is it possible to see every citizen of a certain nation in a list?
"Every" is relative, as eRepublik got more than 9 million accounts. But right now you could use the ranking system at https://www.erepublik-deutschland.de/en/tools/playerrankings to see every citizen.. in a more or less beautiful list.
Mate, please boot Alex Bonte, that mtf that do nothing week after week, month after month, only sell packs, and take over erep labs...
Seriously, I'm so pissed that the luck of the owner is that I don't live in the same country.
I read all you articles and I was given the impression that all of the erep staff work a lot....
Weel, if they do, they are the most incompetent game staff I have ever met, and their productivity is near ZERO. Week after week, the same game shit, no missions, no tournaments, no improvings, nothing we demand.
The only shit they do is change the pack presentation or add some pop up window.. related with what? Packs, good answer.
+1 That's right! But... you still buy packs 😉 only way they will start doing something is to cut down their income... look at my profile... free at last 😛
RIP erebuglik-dutchland.nl
also tell ur BFF Paltons to add to his website exceptions, cuz this: ARE U FOKIN SURE M8 THAT U WANT TO LEAVE UNSER SITE, CUZ DIS LINK M8 BE DANGERUS stuff is a bit annoying, especially if u dedicate too much time for this dying game 😞
also thanks for dedication
also get some beers with my 25cc
No point in what you are doing this game is dead
Why do you even bother talking to those useless developers?
Thank you for your work. I still enjoy the game and use your site. I look forward to the new one.
I have understand about 10%, but well, whatever you say, good job 😛
waste of time
you spent more time on your erep tools than eRepLabs on this game
Game over 🙁
Why exactly do you keep using Apache (with mod_php I guess) for PHP instead of Nginx FastCGI + PHP FPM?
I am currently using apache + php-fpm for dynamic and nginx for static content. The reason behind it is more or less simple: I got a very solid configuration for it.
I already tested nginx + php-fpm but didn't find that optimal settings yet. Furthermore I end up with a quite bigger amount of configuration by just using nginx as there are still several special circumstances you have to keep in mind.
Also: I love my .htaccess file 😛
But seriously: I hope I am able to switch to nginx completely in future, but still: It takes time as I am not a friend of half-baked solutions.
Good bless eRepublik-Deutschland.de!
Keep up the great work!
o7
"This “issue” is caused by the fact that eRepublik evolves"
You made me laugh here..but good luck 🙂
Deutschland, Deutschland uber alles!
*über
o7
if you buy erepublik ill start to pay and play again, but pls first fire all "workers" in erep labs.
[removed]
but pls first fire all "workers" in erep labs.
+1
V
V
o7
great job man, pls don't exclude windows phone app as most ppl do.. 🙂
Keywor😛 Windows Universal App.
Already tested it with another services of mine (http://app.myrustadmin.com) which works quite good (even though you can't find it on the app store yet)
PS: Dear moderator, please don't give me any fp because of external advertising 🙁
Glhf!
i still know some of these words
can't wait for it.... release it!!!!!!!!!!!!!!
v
*Autistic Screeching*
*hdgdl*
o7
Would be nice if I could use the article editor without prior registration!
The problem is, that you should be able to save your articles. I can't do this as long as you aren't logged in as the system doesn't know "who you are".
There are ways to save an article right in your browser (and I may try it out) but there's one big problem: The available space is limited in most of the browsers. Also: You wouldn't be able to access your articles with a different browser.
where is our potato?