Project Direction

Mar 11, 2011 at 6:37 PM

Posting this to try and instigate a discussion on the general project goals for TechCraft. Rather than having an adhoc development adding features here and there i think it would be worth having a think about what we're aiming to achieve with this engine.

Personally i think it would be a good goal to have the TechCraft project develop as a generic engine which could be used for anything. This would basically mean moving any code which is game specific into separate libraries and developing a plugin type interface to allow individual games to override functionality such as region generation, player controllers etc.

Any thoughts ??

Mar 11, 2011 at 6:43 PM

I agree with this idea. I had an initial look at this project and found it really impressive. However, after dabbling with it for a little while I found it too tightly bound to the elements that weren't related to the engine itself. As a result I was relegated to using your engine as an instructional guide more than a useful tool, for that I'm extremely grateful!

If this were to be a generic implementation with clear hooks to allow for easy integration into any other project that would make it infinitely more appealing, to myself that is.

Mar 11, 2011 at 7:11 PM

Definitely follow this approach if possible.

It opens many possibilities for the project to gain additional support. The engine itself is still in very early stages of development, and such a goal taken now, would save a rewrite or two later on to accommodate such features.

An external XML type of configuration could allow for various engine options. An API also for access to functions. Some type of REST support for external communication with the engine. Such features could open the engine to many reuses in other projects.

Mar 11, 2011 at 7:39 PM

I think we already have the concept, we only need to do some clean up :

  • TechcraftEngine project : generic engine
  • Techcraft project : concrete example / demo / reference implementation

Cleanup N#1 would be moving gameclient.cs from engine to techcraft project , maybe even into game.cs

I would like an engine made from extracting reusable code from concrete projects, we could leverage the mercurial fork mechanism here on codeplex :

  • When you start your project, you fork the full techcraft source, and you try to code only in your version of the TechCraft project, you try to not touch the engine.
  • When you really need to modify the engine you can emit a pull request for us to incorporate your engine changes, or send a patch if you want to keep your source closed.
Mar 11, 2011 at 9:03 PM

I've discussed this with some people offline but i can't spend the time on this project to guide it in any way so i've given Enomi overall control..i'm happy to chip in my thoughts and contribute where i can but it's not my baby any more and it's up to you guys to use it or not as you see fit. I will be around to give any help i can but where you go with it is your choice :)

Mar 17, 2011 at 3:32 AM

Just to keep you informed, I m currently trying to rewrite the engine from ground, based on my previous work in java + current codebase + some other samples and articles about cube worlds.

Expect some commits in 2 weeks - I will work harder when I will have beaten dragon age 2 ;)

In the mean time if you have some patches or anything relevant, do not hesitate to send it or just fork the source, I will look !

Mar 19, 2011 at 11:46 PM

Nice one enomi - let me know if there is anything i can do to help - i'm keen to keep involved in some way if i can - i'm still struggling to get the performance of my region loading code (the stuff i sent you) to work with the scale of blocks i'm trying to use in my DBP game... Happy to post over anything i come up with though if it might be of use here...

If you're doing a full rewrite - a couple of people have mentioned to me that you can actually get better performance from this type of engine if you use DrawUserPrimatives rather than building a vertex buffer with SetData. I've noticed in timings from my game that my code to build a vertex buffer which is basically the SetData call takes as long as it does to construct the vertex list in the first place. Something like 120ms to build the list and another 120ms for the SetData call (which also calls list.ToArray() which may have an effect, but i've read that this just does a simple mem copy so shouldn't be significant) for a 96x32x96 region. (XBOX timings, on the PC it's down to less than 20ms on an onboard GPU). Could be worth doing some initial tests chucking a big vertex list/array at both and see which comes out best. I was going to try it, but i'm using Sunburn lighting and can't since it only supports building scenes from vertex buffers.

Mar 20, 2011 at 5:16 AM

DrawUserPrimatives - You already mentioned it, this is in my bookmarks / todo :

But then I found this : : Batching draw calls seems to be so efficient that drawuser vs draw becomes insignificant .

Other subject : I managed to have a block struct storing the lightAmount and the blockType in one byte, so you can have 16 block types with 16 shades of light , or 32 block types and 8 shades.

I think memory consumption may be even more a problem than processing power on the xbox, and this is the reason I m doing a rewrite : I want clean enough code to change fundamental things without breaking everything, and this implies separating the model ( chunk, world, block )  from rendering. I will look into sparse arrays too, we loose so much memory storing all these empty blocks in multiple arrays !

Mar 24, 2011 at 4:25 AM

breaking news ! The rewrite is in good shape, the basics are here (clean model view separation,  optimized drawing of blocks and moveable camera) . I have finally been able to use the code snippet from Bamyazi to simplify block faces removal and optimisation ( no more faceinfo class and associated big array of faceinfo).

Next steps :

1 - reimplement adding & removing blocks

2 - implement dynamic chunks ( infinite terrain )

3 - reimplement trees & water 

4 - reimplement lighting

5 - reimplement collisions

6 - ???

7 - Profit ;)

I will commit something next weekend (at least #1 , I hope to have #2)

Here is a sneak peek at the classes structure :

│   Game1.cs
│   Program.cs
│   Tests.cs
│       FirstPersonCameraController.cs

│   │   Block.cs
│   │   Chunk.cs
│   │   Vector3i.cs
│   │   World.cs
│   │
│   └───terrain
│           FlatReferenceTerrain.cs
│           IChunkBuilder.cs
│           PerlinSimplexNoise.cs
│           SimpleTerrain.cs
│           TerrainWithCaves.cs

    │   Camera.cs
    │   ChunkRenderer.cs
    │   FirstPersonCamera.cs
    │   WorldRenderer.cs

Mar 28, 2011 at 2:59 AM

Rough version of the rewrite is now commited ( newTake project)

I think the code is now more easy to follow, of course it's far from perfect and it lacks of standardizartion ( public fields,  undescore notation vs use of this. etc ... )  but I have been able to correct some little mistakes like having one copy of the texures uv mapping for each chunk, and I reimplemented the block faces removal.

There is block adding and digging ( mouse only, not event tried to support the xbox for now) , directly in the game1 class ( I did not design any player class, but it will come  )

And there is a first prototype of infinite world  : Remove a block on the X increasing direction ( try all sides ! ) and you ll get a new generated chunk !

 No multithreading and just one chunk, and only on X increasing direction but  if you remove another block at the end you ll get one more chunks, rinse and repeat untill memory is full !

 Chunks are stored in a 2d sparseMatrix, implemented with a dictionary<int, dictionary<int,Chunk>>. It's interesting because you can now add new chunks anywhere in this 'infinite array' without any array ranges copies, while keeping coherent coordinates ( so array position is a factor of chunk position in 3d space) . In the current commited code you can even have negative coordinates like viewableChunks[-1,0] but i plan to replace int with unsigned int and start at uint.MaxValue / 2 ( or maybe ushorts, 65535 * 65535 chunks is enough i think ! ). Negative coordinates mess the block picking and require some Math.Abs() in a lot of places, its annoying. 

Having a custom data structure for chunks instead of a plain array opens the way to some crazy gameplay ideas, like circular worlds, other planets (with a 3d sparse array), other dimensions ( 4d sparse array ;), and some good technical ideas too like compressing data of far away chunks instead of just nullifying /  dumping to disk.    

read the code, it will be more understandable ;)

btw it's so rough that there may be mouse grabbing issues and you can only quit with alt f4, but i did respect my  objective of commiting something this weekend so enjoy !