Applied Energistics 2 (AE2) is a mod by AlgorithmX2 which adds very effective and revolutionary storage capabilities to the world. It is so versatile and functional that is has become a staple in nearly every modpack out there. Digital storage! Everything you own can go into a computerized system and gets stored in mass quantities on storage cells you keep in your drives. No more chests, no more barrels, what a great concept! Everything you have to build can be automated, every machine you run can be plugged into it! It's really beyond awesome!
Unfortunately, its versatility also makes it one of the prime sources of performance degradation (lag) on every server it is on. As our bases get bigger and bigger and our production capabilities grow along with the number of items being constantly pumped into our AE2 system it slowly gets worse and worst. Eventually you become the source of lag on your server.. Then what do you do?
The good news is that a properly designed system, even a big one, won't effect performance as much as bad ones and bad ones can always be changed or fixed to work better. This your guide to making that happen.. And it all comes down to three basic rules...
Proper AE2 System Design Rules: Edit
- All items are stored in one (and only one) place
- All devices on the network need to have one (and only one) possible route to the controller
- All external storage devices need to be prioritized, dedicated and have overflow protection
Okay, that all sounds easy enough. But there are a lot of things that need to be looked at and thought through for all three rules to be satisfied.
Stored in one place: Edit
So this might seem like it happens by itself. In the beginning it does, when you don't have a lot of stuff. But as soon as you start running a quarry you start running into problems. Drives get full and things start to get stored in multiple places. You start to see red on the storage cells indicating you have filled it up completely. Now you have the same item in multiple places. and as more items are pumped into the system, the system has to figure out where it is supposed to go.
Your job is to make sure the system never has to figure out where anything goes. Items you know you will get in the 10s of thousands should really have a dedicated storage device. You can use a pre-formatted cell in the ME Drive but it is usually better to just give it a dedicated DSU, Jabba Barrel or Storage Drawer. All three have the advantage of being able to either handle overflow (see below) or in the case of the DSU have such a high storage limit you can leave it alone for your entire career on the server.
In truth you don't need massive storage cells in your system at all. Even the largest bases can easily get by with 16k cells as long as you are handling the high volume items outside of the system. An easy way to tell is just to look at the storage cells every so often. If you are using all the slots but only using 50-70% of the space then you still have room and the system can manage. But you should react whenever they start to get 100% full.
Any time a storage cell goes in the red (totally full) then you should automatically know you have a problem you need to fix. Time to upgrade to the next size storage cell - or take out the item that is sucking all that space and put it into external storage. You can always connect those external devices to the network via a Storage Bus and you can even make it efficient (see below). Just remember to flush those items out of the ME system as you are setting it up so nothing remains in a drive to confuse the network.
One route to the controller: Edit
AE2 devices auto-connect. This is a great feature. But it also means that they connect even if you don't want them to and you can get cross connections. Sometimes these connections will still work and everything will look fine. But the system now has to figure something else out - and that takes computational time for the server.
Any time there are two routes to the same storage block the network has to determine the best route it should use for that packet of items. individually that decision is made pretty quickly. But having to make that decision 50 times a second for hours on end because you are pumping tons of cobble and ore into your setup from 3 Ender Quarries will turn into a mountain of little decisions very quickly. And that mountain becomes lag.
Because the different network components auto-connect this is really easy to get wrong. Most commonly this happens behind a large array of already connected Drive Bays where one cable will run across several of them in a row. It is also common to see this when the same cable runs in a small circle to connect 4-8 Devices that form a small wall. This causes the system to make minor routing calculations for everything that goes into them and if you are processing high volumes of items those minor calculations will add up. Carry a stack of cable anchors around with you and it you think there is a routing issue then just stick an anchor down to correct it. If it would work with an anchor between two cables then it should probably have an anchor there. And if it turns on without a cable physically connected to it then it usually doesn't need it..
Dedicated and prioritized external storage with overflow: Edit
Most people get this petty quickly. Especially since it is an extension of the first rule. Where it typically causes problems is on the priority and on the overflow protection.
Configuring your ME Storage Bus is really very easy. If you're using a dedicated storage block (like a DSU) then you just click the two wrenches on the Storage Bus' configuration UI.
The first wrench sets the item type so the network knows in advance that this item is stored in that device. This is important because if the network doesn't know up front then it has to figure it out while the item is in process.
The second wrench allows you to set the priority for that storage. The default of the system is 0 (zero). So anything over zero, even 1, is better. Not the system is always prioritizing storage even when is not changed. If you have 2 chests and one is closer then it will use the closest one. If the one further away is given a bit more priority then it uses the farther one. The distance is always part of the check routine though and it has an effect on the priority the controller assigns to the routing of the item.
So for large systems that have long cable runs using 1 as a priority really ins't enough. My advise in game is always to think in the 100s when setting priority. All permanent, dedicated storage should just be given a moderately high priority level (like 100 or 200) to ensure that the cable runs and future sub-networks don't interfere with them.
Lets take a common example. You have 7 DSUs plugged into your network on storage buses to process most of the common quarry blocks away from the disk drives into a better storage mechanism. Those DSUs are on the other side of your base just to put them out of the way or to hide them. That chunk is a couple chunks away from the ME Controller that runs the system. If you use set the priority 100 for all the buses then you can be sure that it will always use those DSUs for storing those items as long as they are connected to the network.
The last topic for dedicated storage is "overflow protection". This basically means that your system will deal well when the dedicated storage device runs out of storage room. DSUs have such a high limit that most people would never exceed them so they are considered inherently overflow-proof for most players. For Jabba Barrels and Storage Drawers or similar mass storage blocks though you want to leave some upgrade space for a Void Upgrade. Nothing else really qualifies..
Inputs and Outputs Edit
Thought also needs to be put into how you are pumping items into or out of a system. A bad design sitting idle is usually not a problem. But a bad design that is pulling or pushing multiple stacks per second of blocks is going to cause havoc on a poorly designed network.
The typical setup for a quarry is to pump all the goods into an Ender Chest connected to one ME Input Bus and then let the system figure it out from there. And this works most of the time. But then you add a couple more quarries and an an 8 spawner mob grinder and 8 automated speed farms and now you have a lot of volume getting pumped into the system via that same chest and input bus. If your system doesn't follow the above rules then you will definitely have lag. But even if you follow the rules you may still find the volume is too great. So then what?
The answer depends on where is the volume coming from. If it's a quarry then you should filter out the cobble or stone before it gets to the input bus. If it is a massive tree farm the you want to filter out the logs. Just because you have this super flexible ME system doesn't mean you should use it for everything!
the same applies to the output side. One of my favorite things in most of these mod packs is watching a massive Hungry Node eat up crafting tables and other assorted goodies. I could sit and watch it for an hour just waiting for the numbers to rise. Then you add another pump and another and attach them to banks of assemblers all pulling logs from the network to make wood and then pulls wood from the system to make crafting tables and then export all of them as fast as the server can handle it. Well you have have created a massive lag generator. And in this case the only real solution is simply to do less... Large complicated systems with tons of massive automated assemblers will always crush server performance. So do it or don't do it.. But just know the consequences of what you are doing and how it effects everyone else..
Special Designs: Sub-networks & P2P Tunnels Edit
Subnets are a special class. The truth is subnets exist to improve performance and storage efficiency. The problem is if you set them up wrong then they can actually be horrible lag generators. So use them sparingly and work with someone who really knows how to set them up before you try and use them for mass production.
P2P Tunnels were also added to improve network design and configuration. The number of machines and devices you can fit onto a single network is actually mind boggling. If you really need an ME system that can handle 7,332 connected machines and devices (personal record on SP) then let me know and I can tell you how to do it. But not on a public server.. Excess for the sake of grandeur or ego is really what causes most lag problems in the first place.