Unreal Engine 4

Mixamo animations to UE4. How to import them without support.

In December 2016, Mixamo decided to pull support for Unreal Engine 4 compatible animations. But what does this mean ? Does this make it impossible to get mixamo animations into UE4 ? In short, “No!”, it is still possible to get Mixamo animations into Unreal Engine 4. First I will explain why mixamo needed a separate option for unreal exports and afterwards I will describe a (at the time of writing atleast) working way to get mixamo animations into UE4.

The previous way of getting animations into UE4 would have been to export the mannequin from UE4 as an fbx file, upload this to mixamo, select the desired animations on the mannequin and download them from mixamo with the “unreal Engine 4 export” option. This was however made unavailable.

Mixamo had the need for this option because animations in mixamo work without an actual root bone. I don’t exactly know why, but I guess it isn’t necessary as the preview “scenes” only have the specific character and thus only 1 skeleton. At export time however, the skeleton gets a root bone. When selecting the UE4 option, they added a root bone at 0.0.0 via the way of a script. When you use the standard export, the pelvis gets transformed in a root bone, and this is where the problem lies. UE4 needs a root bone at 0.0.0 to correctly use root motion, and the root bone also describes the position of the mesh. Importing skeletons into UE4 which have the pelvis as root bone therefore results in meshes which “float” by means of the pelvis, resulting in the added effect that a pelvis will not move.

43f29a570e21b0bdbf44e52beffb4e37
A mixamo animation imported with a faulty root bone

Now, for a way to actually import them correctly. First, we must modify the UE4 mannequin to have the pelvis as a root bone (I know , this sounds counter-intuitive). So, export the SK_Mannequin to an fbx. and open it in any modelling software you are familiar with (Maya is sure to work, you may also need to put the model into y-up). Then, remove the “root” bone and the IK bones for the left and right hand. This will leave you with a normal skeleton which has the pelvis as a root. We will call this skeleton_noroot to be clear.

Our skeleton_noroot is now ready to be uploaded to mixamo. You can now put any mixamo animations you want on it. Download the skeleton_noroot with the animations from mixamo in fbx and with a skin attached. Now, import this into UE4, this will give you a new mesh with it’s own animations, materials and skeleton, don’t worry, we will make these animations available on the standard mannequin.

The last thing we have to do is use the UE4 “retarget manager” to retarget our animations from skeleton_noroot to the standard mannequin. This is fairly trivial, as the bones have exactly the same name and the skeletons are actually identical except for the root and IK bones. More information on the retarget manager can be found on Epic’s site.

e20ada53dac32f095c8b0a6fd22f7604
The same animation, correctly imported from mixamo
C#, Code, Unreal Engine 4

Unreal Engine 4 , blueprints or C++?

Unreal Engine 4 is probably one of the most powerful and widespread engines in the industry right now.

For artists it’s easy to see why , it has some very good rendering capabilities, has great functionality with substance painter and is very easy to create new shaders in with its very robust node-based approach.

Programmers new to UE4 however are not always sure how they should feel about UE4. Sure , the blueprint system seems like a nice idea , but can I really do everything I want in it?

Most programmers have a certain fear of the blueprint system at first because they feel like it limits them to what they can do. It also isn’t exactly very freeform at first. Sure , it’s easy to grasp , but very hard to actually implement complex systems in. Atleast , that’s what they think.

As a programmer myself however , i understand all these questions and had them myself at first. Coming from unity , this gives you the idea that you can write everything you want very easily, as it’s totally code-based. Blueprints however are also very powerful , and not should not be underestimated. It’s actually very easy and possible to create a fully working FPS with an inventory etc. without even touching a single line of code in UE4. And all that , thanks to the power of blueprints.

An added benefit of blueprints , is that they can be made extremely readable and it’s a lot easier for the artists on your project to try and code ( or at least understand ) something themselves. This , because it is  essentially the same node-based system as the way in which they create their shaders in UE4.

And when you are a small team , it can be very helpful if an artist ca roughly code one of his ideas and then just let you refine and actually implement this feature.

Therefore , I highly recommend leaving the C++ code on the side and at least trying some code in UE4 blueprints. You’ll be amazed at what it can do.

I’ll be making another blogpost containing some actual examples of me using this system to great effect in the near future.

Stay tuned!

 

 

C#, Code, Diablo 2

Understanding the diablo 2 save file format / Part 3

Now , for the third and final part , let’s look into the checksum a bit , i’ll try to keep my code simple and short this time , so you don’t get a headache while trying to read this post 😉

On a sidenote , this contains a lot of bitshifting etc. which is not ideal to write in c#, but verything should still be straightforward enough to translate into another language. 🙂

The checksum is actually pretty simple , first of all , you set the checksum bits to 0. ( they are bytes 12-15 )

character.BinaryStream[12] = 0x00;
character.BinaryStream[13] = 0x00;
character.BinaryStream[14] = 0x00;
character.BinaryStream[15] = 0x00;

Now for the checksum algorithm. This one is also fairly simple. All you have to do is add up all the bytes one by one and everytime you add a byte , you bitshift the sum 1 spot to the left. You do however have to make sure to carry over the highest bit( most bitshifting operators do not do this , so we have to do it manually ) , as we do not want this value to get lost , so we actually just rotate our sum around 1 bit everytime we add another value.

uint checksum = 0;            

for (int i = 0; i < character.BinaryStream.Count(); i++)
{
 byte thisByte = character.BinaryStream[i];

 uint carry = 0;
 if ((checksum & (1 << 31)) != 0)
   {
    carry = 1;
   }

checksum = unchecked (unchecked(checksum << 1) + character.BinaryStream[i]) + unchecked(carry) ;
}

And now , we just have to get the checksum back in the 4 bytes!

 byte[] intbytearray = BitConverter.GetBytes(checksum);
character.BinaryStream[12] = intbytearray[0];
character.BinaryStream[13] = intbytearray[1];
character.BinaryStream[14] = intbytearray[2];
character.BinaryStream[15] = intbytearray[3];

If all went well , you should now be able to read in files ,save them out , and still be able to play them !

C#, Code, Diablo 2

Understanding the diablo 2 save file format / Part 2

This time , we’ll be taking a look at how to read in the statistics bit of the d2 save file. First of all , we’ll need to establish how we will be reading in this part , I suggest copying this part in a separate bit/bool array for easier reading seeing as this is not byte aligned anymore.

First, we check out the length of the statsheader by looking at the location of the skills header . We also know for sure that the statsheader starts at 767 due to the fact that all files are formatted identical up untill this point.

int statstart = 767;

for (int i = 765; i < fileSize; i++)
{
    if (fileStream[i] == 'i')
    {
        if (fileStream[i + 1] == 'f')
        {
            skillHeaderStart = i;
        }
    }
}
int statlength = skillHeaderStart - statstart;

Now , let’s put the statsblock in an array of bools for easier reading ( so we dont have to skip over to another byte everytime )

List statBitStream = new List();

for (var i = 0; i < statlength; i++) // Getting the statlist in a bit array
{
  for (var j = 0; j < 8; j++)
  {
    var bit = GetBit(fileStream[statstart + i], j);
    statBitStream.Add(bit);
  }
}

private static bool GetBit(byte b, int position) // position in range 0-7
{
 return (b & (1 << position)) != 0;
}

Now that we have gotten individual bits in an array , let’s try reading them , and converting our values. The values always have an “id” part and a “value” part , the id part always consists of 9 bits and contains values between 0 and 15 ( meaning there are at maximum only 16 stats contained in this block ) . The size value part’s size however , depends on the particular id.

Now , for id’s and their values’ size :

private int StatIdToBitCount(int id)
        {

            int bitCount;
            switch (id)
            {
                case 0:
                case 1:
                case 2:
                case 3:
                case 4:
                    bitCount = 10;
                    break;
                case 5:
                    bitCount = 8;
                    break;
                case 6:
                case 7:
                case 8:
                case 9:
                case 10:
                case 11:
                    bitCount = 21;
                    break;
                case 12:
                    bitCount = 7;
                    break;
                case 13:
                    bitCount = 32;
                    break;
                case 14:
                case 15:
                    bitCount = 25;
                    break;
                default:
                    // UNKNOWN BITCOUNT
                    bitCount = 1;
                    break;
            }
            return bitCount;
        }

With this out of the way , our program will recognise the amount of bits he has to read for the value of our particular stat. You will notice i did some weird shenanigans to properly read in the values ( namely multiplying the bits times 2^x depending on the bit position )

int streamIndex = 0;
bool doLoop = true;

while (doLoop)
{
    var id = 0;
    if (streamIndex + 9 > statBitStream.Count()) // stop when end of stream is reached;
    {
        doLoop = false;
        break;
    }

    for (int j = 0; j  statBitStream.Count() -StatIdToBitCount(id)) 
// stop when end of stream is reached;
    {
        doLoop = false;
        break;
    }

    uint valueForStat = 0;

    for (int d = 0; d < StatIdToBitCount(id); d++) // Add bits to the valueforstat
    {
        int addedvalue = 1;

        if (d == 0)
        {
            if (statBitStream[streamIndex + d]) { addedvalue = 1; }
            else
            {
                addedvalue = 0;
            }
        }
        else
        {
         for (int i = 0; i  statBitStream.Count()) 
// stop when end of stream is reached;
        {
        doLoop = false;
        }
}

This post is mainly a big wall of code , it may be rather ugly to read . But I do find it important for people so they can see how they would go about translating the information about a binary file into actual code that reads this.

So , that was it for now , next post , and probably the last about the d2s file , will be about manipulating the checksum so you can actually save your file and play your manipulated save in the game.

C#, Diablo 2

Understanding the diablo 2 save file format / Part 1

First of all , we need to understand the basic structure of the file , which consist of several parts each with their own header.

First we have the main header , which contains basic information such as name , class , which patch the character is on , is it a HC character? etc…

The second part is the quest completion data , which starts with the header  Woo! . On a sidenote , when setting future waypoints active which are located in acts you have not yet visited I’m pretty sure you need to have completed the previous act’s main quest.

Next up is the waypoint data with the WS header. This part is a bit more tricky as the data for the waypoints is stored in 5 bytes , which means you need to check if these bytes’ bits individually to see if they are set thus extracting the byte array to a bit array  and then modifying them to store them in a byte array once again ( this isn’t necessary , but is in my opinion a cleaner way of managing the data in your application as an array of bits ( bools ) rather than constantly managing a byte array.

The third part isn’t such an important part , it just contains the information about introductions being made to certain npcs or not. the identifier is w4.

Now comes arguably the hardest part of the file , the statistics part. Up until now ,the structure is identical for every file , meaning the byte offsets are always the same and can easily be indexed. These stats however , vary from file to file. There are 16 statistics in total , but not every statistic is always present. Meaning the size of this block varies, this block also isn’t byte-aligned anymore , giving you even more of a headache. The exact way to read this file will be explained in another blog post with some example code.

Next up is the skills block , this is fairly trivial to read. It starts with the header “if”. This is followed by 30 bytes which correspond with the points invested in your character’s skills (meaning if you have an item equipped which boosts skill points , this will not be reflected here.) The correct indexes for every skill , depending on your class can be found here .

The last major part contains all your characters items ( equipped, in belt , in stash , in inventory… ) and starts with “JM” , followed by a short , which represents the number of items. After this there will be other headers similar to this one (starting with JM , followed by a short ) . The first ones short will be either 1 or 0 , if it’s 1 , your character is alive and there is nothing special. When it is 0 however , this means your character is dead  and thus has a corpse. 12 unknown bytes will follow , and after this another item header , this time containing the items which can be found on your corpse. If you have an expansion character , this will be followed by the identifier jf if you have a mercenary. Potentially followed by another item header , containing your mercenaries items. This item list will finally be closed by “kf”.

Oof , that’s it. A lot , I know , but a necessary evil nonetheless. More information on the exact indexes can be found here .

I’d like to thank both xmission and coreyh for putting these handy resources out there as they have been a great help to me and a multitude of others. If these resources ever become offline , do not hesitate to contact me as I have tried to conserve these offline myself if something ever were to happen to these pages.

Now , on to the next post , containing some vital information on reading the character statistics.

C#, Diablo 2

Diablo II save file editor / C#

As a long time Diablo II fan and with the 20 years anniversary being this year , I decided to try and make a save file editor .

I certainly wasn’t aspiring to reinvent the wheel though , there have been handy mods for this in the past , and with the full respec functionality that plugY offers it isn’t that handy of a tool either. Nevertheless I found it an interesting and challenging exercise in loading binary files.

Because in my search for information about the .d2s file format i stumbled upon a lot of dead ends and scattered resources , I will try and combine all the necessary resources to read the files right here for any other dabbling coders.

The application was ultimately made in wpf for an easy UI and to implement the MVVM model , so all code was done in C# , which made reading in Binary files a tad more cumbersome. If you really just want to read in the files I would rather recommend using c++ or c. Also , when trying to read the file , do try and puzzle the pieces together yourself , I’m sure you’ll find it interesting.

Now, with this introduction aside , I’ll split this up in multiple posts on my blog for easier access to the parts you need / are interested in. (This blog has tags , feel free to use them)

diablo-ii