27

I would like to know just how much disk space I will need for such a world in Minecraft. It will be stored on USB which has 3.12 GB of free storage remaining. This is for a Minecraft server. This world would be the size of normal Minecraft excluding the non-existent Far Lands.

Vemonus
  • 63,784
  • 66
  • 267
  • 403
Phoenix
  • 515
  • 1
  • 6
  • 15

6 Answers6

44

Are you trying to build the Earth (it's actually about 22584607 by 22584607 metres surface area, about 3.2 times what you're asking about)? Well, the block height can take 256 values and each block uses at most ~2.5 bytes, so it's 1255082412550824256*2.5 = 100,814,837,170,544,640 bytes, aka 100,815,000 GB, aka 101 petabytes. You're going to need a bigger disk ;)

Compression could reduce this somewhat, but depends heavily on the complexity of the world. So unless your world is ultraflat it's probably still going to be a petabyte or more.

Community
  • 1
l0b0
  • 3,110
  • 2
  • 31
  • 47
  • 5
    Did you factor in the compression? It was mentioned in the link. – Phoenix Apr 21 '20 at 04:55
  • 1
    I'm not sure exactly how it works, but if I were to make minecraft (and I have reason to believe the people working on Minecraft might know what they're doing better than I do) then I would only save chunks which have changed since they were generated by the terrain generation algorithm. In that case, the total size of the map doesn't matter, only how much you have changed the map. – Turksarama Apr 21 '20 at 07:05
  • 4
    @Turksarama You'd need to (re-)generate millions of blocks per second as the user moved around, which would be slow and expensive. I'm pretty sure the current solution is close to optimal. – l0b0 Apr 21 '20 at 07:26
  • 9
    As soon as a chunk gets generated, entities start moving in that chunk, lava and water start flowing, bamboo and sugar cane start growing, lots of other things happen, so there's changes to the chunk all the time. Which means the chunk needs to be saved; there just aren't any pristine generated chunks in Minecraft. – Guntram Blohm Apr 21 '20 at 09:46
  • 3
    Even though the block height is 256, the a chunk is split into 16 pieces vertically, and parts that are fully filled with air are not stored. You are also using an outdated figure of 2.5 bytes per block, as newer minecraft versions use a "pallete/index" based system, where you have a pallete of blocks per 16x16x16, meaning that the space a block takes up is greatly reduced. (4-5bits per block is typicall, with some spikes for chests and other advanced blocks). You also have to factor in that storing the pallet takes up space – Ferrybig Apr 21 '20 at 14:52
  • Plus, if you are trying to build the Earth, as this answer speculates, the procedurally generated chunk is almost never what you want anyway. – TheHans255 Apr 21 '20 at 14:56
  • Im not trying to build the Earth. But does the current size of let's say 101 PTs factor in the compression that minecraft uses as mentioned in the answer in the link? Use average compression on a perfectly normal world generated by Minecraft, plus with a bit a man-made structures (A wooden house with a bed, and a chest). – Phoenix Apr 21 '20 at 17:55
  • @GuntramBlohmsupportsMonica Does that mean every time you save that chunk with entites, it is a record of where all entities were, and will they also get compressed? – Phoenix Apr 21 '20 at 18:00
  • 2
    I don't think trying to get much more accurate is helpful. @Phoenix asked about 3 GB, and the answer is clearly several magnitudes away from that. – l0b0 Apr 21 '20 at 20:14
  • 2
    @Phoenix Yes, a chunk has a list of entities in that chunk, and when saving the chunk, this list gets saved as well. Each of the region files (in world/regions/r.X.Y.mca) has a matrix of 32x32 chunks, and each of the chunks gets individually compressed in that file. – Guntram Blohm Apr 21 '20 at 21:08
  • 1
    @phoenix Even with the most amazing of all compression rates of say 90% this going to be way out of a normal computers league. Realistically you could get maybe a 100 TB array for a non ridiculous price (and we're still talking thousands of dollars here) (16x 8TB) and that wouldn't even put a dent into the sums required. – Voo Apr 21 '20 at 21:40
  • 9
    In my experience, most chunks in Minecraft are usually either 8 or 12 KiB once compressed (the Anvil file format pads chunks to a multiple of 4096 bytes in order to improve the efficiency of disk access). That would put the size of the world somewhere around 2 PB. – NobodyNada Apr 22 '20 at 04:11
  • 1
    It costs about $400k to store 1PB for a year. That's one million dollars to run this server for a year. – Mazura Apr 23 '20 at 00:44
  • @l0b0 someone with terabytes of storage might have a similar question, stumble upon this, and will be greatly helped by the unambiguous answer that no, a few terabytes aren't enough either, even taking all kinds of compression into account – Leif Willerts Apr 23 '20 at 14:07
28

As the existing answers only guess about compression, I estimated the space requirements from an existing server. That server has a 60kx60k world (-30000 to +30000 each X and Z), which is approximately 50% generated, and which needs 28 GB.

Let's assume it needs 56 GB fully generated.

Your world has ~ 210 times the size in X AND Z, so 44,100 times the size. 44,100 * 56 GB makes 24,696,000, so a bit less than 25,000 TB, or 25 PB.

This is way more than achievable with a standard PC, but if you build a supercomputer, it's very well reachable. The IBM Summit has ten times as much, and it's a real bargain at $200M.

Now, let's check how much time you need to generate the world. On my Ryzen 7 3800X, using WorldBorder, I can generate approx. 200 chunks per second. Your world has (12550284²)/(16²) ~ 6.1*10¹¹ chunks. This makes 30 trillion seconds, or 35,606 days. Wow, just 100 years to populate your world after you've assembled the computer to store the data, and started the generation process.

Of course, you'll probably have more than one CPU in your supercomputer. Let's assume you have 64 cores, set up 64 instances of Minecraft, and let each of them generate a part of the world. Assuming the CPUs are faster than mine, this may give you a factor of 100. So your world may actually be done in just 1 year!

Timmy Jim
  • 54,489
  • 32
  • 188
  • 251
Guntram Blohm
  • 1,170
  • 8
  • 13
  • 1
    Just doing the math in my head.. ~10^7/~10^4 ~ 10^3. 10^3^2 = 10^6. So the given map is very roughly a million times bigger than your existing server and not 440 times. I think you got the original world wrong by an order of magnitude. – Voo Apr 21 '20 at 11:16
  • Doh, you're right. When I wrote my answer, I had Odd's answer on the screen, which is missing a digit, not the original post. – Guntram Blohm Apr 21 '20 at 11:21
  • So, that means if I were to teleport to the now non-existent Far lands, it would be tremondously slow on a ordinary computer with an Intel i5? – Phoenix Apr 21 '20 at 17:57
  • 4
    No, because only the chunks you actually visit get generated; teleporting anywhere creates some chunks around the place you teleported to, not the whole world between spawn and that place. – Guntram Blohm Apr 21 '20 at 21:10
  • Large supercomputers have tens of thousands of microprocessors, so it is conceivable that one could generate the world in a day, given the rest of your analysis is reasonable. – Lawnmower Man Apr 22 '20 at 01:12
  • $100 000 for 1PB of storage; https://www.youtube.com/watch?v=C2uLSOmRx_c, so 25PB should be achievable for not much more than $3M. (Of course not having a supercoputer is going to hurt when it comes to populating the data) – Taemyr Apr 22 '20 at 09:42
  • Now do electricity cost... – Leif Willerts Apr 23 '20 at 00:31
  • 1
    Involving a supercomputer (Tianhe-1A is ~4MW per hour, so that would equate to $400 per hour with 10 cents/kWh, which in a day is $9600)? – Phoenix Apr 23 '20 at 01:52
  • 2
    @Phoenix: Jokes about supercomputers aside, it'd probably be cheaper to design a better logic system to meet your goal more efficiently. I mean, I doubt your goal is to actually store such a massive world, as the same problems with storing/generating it apply to observing it. – Nat Apr 23 '20 at 05:09
  • 1
    I think this answer is much better than the accepted one, because it is based on data from a real Minecraft server and not just a guesstimate based on theoretical numbers. – Philipp Apr 23 '20 at 10:41
  • You don't need a supercomputer, but you do need a normal computer with a zillion hard drives attached to it. Basically you want 50 of these servers which is probably going to take up 2 or 3 racks. And a ton of money. – user253751 Apr 23 '20 at 14:29
  • (but you don't need that to go to the far lands) – user253751 Apr 23 '20 at 14:51
6

In regards to the main question of "How many bytes would a world of this size require?"

The answer to that varies between Minecraft version, the kind of compression involved and the specifics of the world being created, and thus is impossible to get anything more than a ballpark figure for.

In regards to the secondary question of "Can I fit a world of this size in the 3GB left over on my USB stick", I'd like to demonstrate a different way of proving that 3GB is nowhere near enough using a worked example and some fictitious assumptions to make the maths a bit more appealing.

Let's assume that:

  • 1 chunk = 16x16x256 blocks
  • 1 chunk requires 1 byte
    • It's probably a lot more, this is a simplification

In which case the question becomes how many bytes would you need to store a 12550824 * 12550824 world...

  • 12550824 / 16 is 784426.5
    • The number of chunks along each dimension
  • 784426.5 * 784426.5 is 615324933902.25 bytes
    • The number of total chunks, and thus the number of bytes needed to store that many chunks, assuming 1 byte per chunk
  • 615324933902.25 / 1024 is 600903255.763916015625 KB
  • 600903255.763916015625 / 1024 is 586819.5857069492340087890625 GB
  • 586819.5857069492340087890625 / 1024 is 573.06600166694261133670806884766 TB
    • The total number of terabytes, stopping there because that's smaller than 1 petabyte

~574 Terabytes worth of fictional 1-byte chunks.

So assuming that 1 chunk could magically be stored in a single byte (a feat that is, as far as I'm aware, impossible), you're already looking at terabytes of information. In reality, chunks would undoubtedly require more than just a single byte of information, so you'd easily be looking at more than just a few terabytes..

Hence the answer is "Not even close". Either shrink your world or get a bigger USB stick.

Pharap
  • 939
  • 10
  • 19
  • 3
    "[O]r get a bigger USB stick." Right, I need to go find my 1PB flash key... Know where I could buy one if I can't find it? – Himitsu_no_Yami Apr 22 '20 at 21:01
  • 1
    @Himitsu_no_Yami If you've got enough money then a house full of 32TB SSDs hooked up to servers might suffice. (Not to mention the energy bill and the liquid nitrogen needed to keep everything sufficiently cool.) – Pharap Apr 22 '20 at 21:14
  • But then it's not a USB stick anymore – Himitsu_no_Yami Apr 22 '20 at 21:15
  • 2
    @Himitsu_no_Yami It is if you build the house in the shape of a USB stick. But they do already make house shaped USB sticks, so I'm sure nobody would know the difference if you just stuck a USB cable on the side of the house. – Pharap Apr 22 '20 at 21:25
4

As other answers have stated, a fully generated 12550824 x 12550824 world can take up to 12550824 x 12550824 x 256 x 2.5 bytes = 1.00815e+17 bytes or 101 PB. If we want to be generous and assume that large chunks of air blocks are not stored, we can probably reduce the effective height to 128 blocks, yielding 50.5 petabytes. However, your world will only take up this much space if every single chunk within that space has been observed by a player.

If, instead of doing that, you spawn at (0, 0) and immediately teleport to (12550824, 12550824) (/tp 12550824 ~ 12550824), you will instead only generate the chunks immediately around (0, 0) and (12550824, 12550824). If your render distance is 32 chunks, this means approximately 32 x 32 x 2 = 2048 chunks will be generated. This yields a total of 134,217,728 blocks, for up to 335,544,320 bytes, or 335 MB - well within the confines of your USB drive. None of the other chunks, either between the two teleport points or anywhere else in those square confines, exist - you've never seen them, so why should the engine go through the trouble of generating them?

Now let's consider a case where instead of teleporting, you simply walk due East until you reach (12550824, 0). This will essentially generate a straight line of chunks around you, reaching up to 32 chunks (or 1024 blocks) on either side of the line, plus 1024 blocks behind your start point and 1024 blocks ahead of your finish point. This means that a rectangular prism of (12550824 + 2048) x 2048 x 256 = 6,581,320,155,136 blocks will be generated, or up to 16,453,300,387,840 bytes or 16.5 TB. Still way too big for your USB stick, but a small amount of compression would probably get that under 16 TB, and you can buy good-quality 16 TB hard drives for less than $500 USD.

As a middle ground, let's consider teleporting through non-command methods - using Nether portals. Each block length down in the Nether is equivalent to 8 lengths in the Overworld, so if you build a Nether portal at (0, 0), walk due East to (1568853, 0), and build another Nether portal there, you will pop out at (12550824, 0) in the Overworld. The Nether will generate a rectangle of chunks as in the previous paragraph, though this rectangle will be eight times smaller: (1568853 + 2048) x 2048 x 256 = 823,604,543,488 blocks, or 2.06 TB. In the Overworld, however, the space between the two portals will not be generated, and you will end up with the two islands of chunks described in the /tp case - only 335 MB, which keeps the effective total data to 2.06 TB. A little compression will get that under 2 TB, and 2 TB drives are even more affordable, with good quality drives coming in at just a little more than $50 USD.

As an aside, USB sticks are notoriously short lived and not durable, and I would not recommend one of any size for running a Minecraft server. I would only consider the option if a) you are only using the stick to transfer the world and are not actually running it off the stick, or b) maybe you are running the server as some sort of one-night, LAN party thing and don't see better hardware worth investing in (though I'll aside in that case that you probably won't explore nearly any of that generated world in that time). If you have access to any better dedicated hard drives or SSDs (even one connected over a USB SATA or NVMe interface), I would highly recommend using those.

TheHans255
  • 435
  • 3
  • 9
2

Alright, so... you have 1255084 x 1255084 blocks x 256 (remaining air blocks or other block u wanna put) and multiply. you get 4.0326038e+15. assuming each block is a byte, you have 4.0326038e+15 bytes, which is a LOT more than 3 GB. so you may want to consider a smaller world. The best USB sticks sold right now for that amount of data are hundreds of USD.

Odd
  • 41
  • 1
  • 8
  • 6
    It's not 1255084, but 12550824 - you are missing a digit. So you might be off by a bit ;) – Timmy Jim Apr 21 '20 at 13:29
  • 1
    so that would technically be almost ten times more than my previous answer. thats a lot of bytes – Odd Apr 22 '20 at 01:11
  • 1
    almost a hundred times in fact. also, hi @Odd , do you not have the privileges to edit your own answer yet? – Leif Willerts Apr 23 '20 at 14:04
  • i can edit my answer, but its only 10x more data than my previous mistaken calculation – Odd Apr 24 '20 at 15:23
-2

I think I might have a solution 1.Get Optifine to help Minecraft load in the world 2.Turn your render distance in down to 2 to unload a lot of chunks 3.Move the world into 1.15

Hyper_Hello
  • 1