Steam RoF Development Tracking

[Noport]

understand now why i wasn't seen star portal in ew you wasn't loading eastwastesshard zone for RoF clients it loaded eastwastes zone

[trevius]

The aggro meter is a new system for RoF, and new systems do not need to be implemented for this thread. The point is to get the client caught up with the functionality of our previous clients. New features and systems may be implemented later, but they are not required as part of our client compatibility development unless it is something that prevents another previously functional system from working.

The Eastern Wastes issue with the new zone is not really part of this thread either, as that is a DB change, not directly related to client compatibility. This thread is for developing the source code not the database unless it is something like a new field that needs to be added to the database (as apposed to a new entry in the DB).

The mercenary timer issue is also part of a new system that is being worked on for all clients, not just RoF. We are still working through the mercenary stuff to make it work 100% as it should.

I believe Derision is still working through the Guild Management stuff to implement the new ranking system changes and other stuff. I am currently looking at the bazaar trader packets to see if I can figure out how to get that stuff working. Once these 2 things are done, the last main thing to work on will be the new slot system. Once the new slot system is in place, we should be able to correct the cursor buffer issue and also implement the new augmenting system. That should pretty much conclude the initial compatibility work for RoF.

It is getting pretty close now, but the new slot system overhaul is a big change and may take a while. Also, I have noticed so far that some of the bazaar changes may be a bit more complicated to get working, but hopefully it won't be too bad. I am still just starting to look into that.

[gibroni]

not sure if this is a problem with RoF with emu only, but i believe so. trying to load a spell set and you already have some spells memmed. it should remove spells and remem the spell set as saved. Says you already have the spell memmed. UF and previous clients would remove spells and remem spells in the correct order they were saved and reload them that way no issues. trying to run a macro i have, also will not demem a spell to mem the new spell. if i manually a spell from the bar, it then will mem and cast. will go load up live and give the reload spellset and see what happens.

edit: it does remove spells and remem them correctly on live.

[sorvani]

During development of the UF client the same issue was experienced.. Interesting that it resurfaced.

[trevius]

It looks like it is just a simple packet struct change. Since all of the gem fields in that packet are basically bools, they changed them from uint32 to uint8. Also, ShowEQ notes that MAX_PP_MEMSPELL was changed to 16, so that is what we have it set to for RoF to use in other structs, but in OP_LoadSpellSet I think it is actually only 12. The packet is 16 bytes, but the last field I think is a uint32, which means it only uses 12 spell slots. The value for the last field I saw was 12, so maybe it is a gem count field? Though, the char I tested it on only had the basic 8 gem slots available.

Here is an example of the packet:

[OPCode: 0x38b4] OP_LoadSpellSet [Client->Server] [Size: 16]
000 | 01 01 00 00 00 00 00 00 01 01 01 01 0c 00 00 00  | ................

And, I think the following changes will fix the issue, but I haven't tested yet:

RoF_structs.h

struct LoadSpellSet_Struct {
	uint8	spell[12];	// 0xFFFFFFFF if no action, slot number if to unmem starting at 0
	uint32	unknown;	//Seen 12 - Maybe a gem count?
};

Note that I had to hard set the spell count to 12, when it normally is set by MAX_PP_MEMSPELL, because MAX_PP_MEMSPELL is set to 16 now. Maybe MAX_PP_MEMSPELL should be changed to 12 in RoF, but that would require a few changes where it is used as well (which may be good or bad).

RoF.cpp

DECODE(OP_LoadSpellSet)
{
	DECODE_LENGTH_EXACT(structs::LoadSpellSet_Struct);
	SETUP_DIRECT_DECODE(LoadSpellSet_Struct, structs::LoadSpellSet_Struct);

	for(unsigned int i = 0; i < MAX_PP_MEMSPELL; ++i)
	{
		if (eq->spell[i] == 0)
			emu->spell[i] = 0xFFFFFFFF;
		else
			emu->spell[i] = eq->spell[i];
	}

	FINISH_DIRECT_DECODE();
}

Also, note that before the server sends the packets to unmem the spells, it replies to the client with the same packet it was sent, per this example:

[OPCode: 0x38b4] OP_LoadSpellSet [Server->Client] [Size: 16]
000 | 01 01 00 00 00 00 00 00 01 01 01 01 0c 00 00 00  | ................

I don't think that packet is required, but it is something to keep in mind.

Either way, this should be a really easy issue to resolve. I will test it today or this weekend and commit if no issues. Thanks for the report!

Edit: Actually, it looks like that would only affect the Player Proflle Struct/Encode, so it probably could be changed. If the location of the mem_spells field is correct in the RoF PP, something is a bit odd. For all fields with multiple iterations in the RoF PP, there is an iteration count field right before it. For the mem_spells field, the count was observed as 16. Though, only 12 fields are set in that iteration and the rest are all 0s. Maybe that just means they are planning to expand to 16 spell gems max at some point and have already added that part to the PP, but not other parts of the code dealing with memmed spells.

[trevius]

BTW, I have been working a bit on figuring out the bazaar stuff. I am still looking through packets and currently just trying to get starting trader mode to work. It looks like they changed the way items are tracked to a serialized id string to keep track of all items individually. I don't think we actually need to use this new system initially, but to match Live exactly, it may be required eventually.

I tested on Live and found that I am able to have whatever I want in my trader satches, and as long as I don't try to sell anything that is no drop, it will let me enter trader mode and just not list the items I didn't set a price for and ones that are no drop. On EQEmu, we just check for the item ID of the trader's satchel and cycle through the 8 inventory slots looking through the satchels. If there are any items that don't have a price set or of any no drop items are in the satchels, it just prevents trader mode from being started. This isn't a major issue at all, but figured it was worth noting here for reference.

Another thing to note is that previously only 80 slots were possible for trader mode due to having a max of 8 trader satchels with 10 slots each. On RoF, it has been changed to 200 slots due to having 10 main inventory slots and I am guessing there is now a 20 slot trader's satchel available so you can reach 200 slots for sale. Since we currently check for the trader's satchel by its item ID, we will need to also add the item ids for any new trader's satchels once the new slot system is in place that will allow larger than 10 slot bags. We will also need to increase the slots searched from 80 to 200. This is something that can be done later when the new slot system is completed and won't affect the initial implementation of the bazaar systems in RoF.

Once I can get trader mode starting properly, I will just keep working my way through the packets to get the rest of the trader system worked out. I think the hardest part may be the fact that you can use /bazaar search from any zone and purchase items directly through that system. It then sends the purchased item to the parcel system, which is another system that will need to be implemented.

This is the first time I have messed with bazaar stuff at all, so it is taking a while for me to get familiar with it and work my way through it. Hopefully I can get it all worked out.

[thepoetwarrior]

Not sure if this was mentioned yet, but just tested RoF and quest npc that was suppose to give back more than 1 item only gave back the 1st item, and the rest of the items were not on cursor. I realize RoF / F2P isn't 100% yet but just adding this one if not known yet.

[trevius]

Originally Posted by thepoetwarrior

Not sure if this was mentioned yet, but just tested RoF and quest npc that was suppose to give back more than 1 item only gave back the 1st item, and the rest of the items were not on cursor. I realize RoF / F2P isn't 100% yet but just adding this one if not known yet.

Yeah, that is a known issue with the cursor buffer. It actually does give everything back, but you can't access the cursor buffer without zoning or relogging. So, if you got 4 items back, you would need to zone 3 times to actually get them all off of your cursor. I think Uleat may have a solution for this in the works. In the mean-time, at least there is no item loss involved, it is just an annoyance to have to zone to get the items back.

[thepoetwarrior]

Oh, thanks for the info! So happy no item loss!

Edit: Found your post on page 5 about augs, sorry for spam

I know I know, I got 8 pages to read here + forum search

[Drajor]

BUG:

Command - #unscribespells

Spells stay in RoF spell book, but can not be mem'd. After relogging the spell book is empty. On UF the spell book is wiped immediately. Looks like OP_DeleteSpell may have changed a little.

[Zia]

While I do not play either on PEQ nor the Test server (I can not seem to find it?) I have noticed a few things on the server I do play on. I hope I am doing the right thing posting them here, otherwise just ignore this post!

1. Going to third person camera and then to first person will make the particles from both primary and secondary slot items show on your screen. This is surprisingly annoying.

2. There are no sounds while casting spells. There are sounds when they are landing though.

3. Maps do not seem to work correctly. I have not been able to pin-point this exactly, but in some zones the map is not displayed. My map for Nedaria's Landing refuses to show itself, for example. This map I downloaded and is not the original map (if there even was one).

4. Charms (ie, Intricate wooden figurine) is empty, as is the Adventurer's stone.

5. Checking out charm slot stats can make you go LD. It does not happen every time, but frequently enough for it to be some sort of pattern.

[Zia]

Originally Posted by Zia

3. Maps do not seem to work correctly. I have not been able to pin-point this exactly, but in some zones the map is not displayed. My map for Nedaria's Landing refuses to show itself, for example. This map I downloaded and is not the original map (if there even was one).

Scrap this. Today it worked fine, for some reason.

[Derision]

Originally Posted by Drajor

BUG:
Command - #unscribespells

Spells stay in RoF spell book, but can not be mem'd. After relogging the spell book is empty. On UF the spell book is wiped immediately. Looks like OP_DeleteSpell may have changed a little.

Originally Posted by Zia

2. There are no sounds while casting spells. There are sounds when they are landing though.

These two are fixed in the latest Rev. First was an incorrect opcode and the second was a changed struct. Thanks to both of you for the bug reports.

[Drajor]

Originally Posted by Derision

These two are fixed in the latest Rev. First was an incorrect opcode and the second was a changed struct. Thanks to both of you for the bug reports.

Winner! Cheers heh

[trevius]

As now noted at the top of the first post of this thread in big red letters, Steam has updated the RoF download to the Jan 16th patch from Live, so it is no longer compatible with EQEmu.

It was a known possibility that Steam might start doing semi-regular updates to the client downloads. Hopefully if they continue, it is at least automated on their end (which I suspect it is), which should mean the dates it gets updated would be predictable. The current interval was about 2 months (RoF was added to Steam on 12-13-2012, and the earliest report of it being updated was on 02-11-2013). If the updates are every 60 days (which lines up with what we know so far), that would put the next Steam download update at about 04-12-2013. If it happens then, that should confirm their schedule, which should make it easier to plan for and make people aware of pending updates.

The EQEmu Dev Team is already discussing our options to make a solution for regular updates. The good thing is they are less frequent and hopefully more predictable than Live patches. The bad thing is it will require regular upkeep from the Dev Team to keep EQEmu working with Steam. As far as I know, Steam is still our best option for an easily and legally attainable client so I think it is worth the effort if we can streamline it.

I can get opcodes updated from patches fairly quickly now. And the ShowEQ team normally figures out the position update struct changes pretty quickly (which happen every patch), so those are 2 pretty big parts of making a client compatible that can be done quickly and easily. There will still be struct changes required here and there and those are not predictable and will have to be worked as they are found. At least by keeping up with them regularly, we shouldn't run into too many major changes for most of the updates which should limit the amount of work required for getting a new client fully compatible. The difference in time between the UF client and the RoF client was pretty long (over 2 years), which is why it is taking so long to get the RoF client fully compatible. By keeping up with the clients regularly, it will mean just a little work is required at a time and the new client will be complete unless there was a major overhaul of some system that requires major changes on our end (which shouldn't be too frequent).

My main concern is how to deal with having multiple versions of the same expansion all compatible with EQEmu. We can easily enough just copy the patch files in the source and make adjustments to the copies. But, that leaves us with potentially dozens of different patches to deal with eventually. This isn't really a problem until we try to implement a new system that we want to give all clients support for. For example, if we eventually add in Shared Tasks or the Expedition system, we would not only have to add structs, opcodes, encodes and decodes to Titanium, SoF, SoD, UF and possibly RoF, but we might also have to add them to 15 (made up number) other version's patch files as well. It would mean the same thing for any packet related fixes that get put in. This is nearly impossible to manage, and would probably lead to new stuff and fixes only being implemented into the latest client, which certainly isn't ideal if it can be avoided.

Another problem is that in order to support all clients properly, the dev team has to actually have all clients and test each one if the change might break something with certain clients. This is bad enough now having 5 or more installs at ~10GBs each, but at 20 installs that is just insane to even consider.

Since each client potentially has packet struct changes for packets that aren't already encoded/decoded, the patch files are just going to keep getting bigger and bigger even if there are only minor differences between the current client and the version just before it. One idea I have had so far (but not really sure how to implement) would be to have mini-patch files that only include the structs and encodes/decodes that differ from the client directly previous to it. So, if only the position updates and spawn struct gets changed, then those are the only structs and encodes/decodes we would need to put in there. If this is possible, it might resolve the issue with having to update multiple patch files anytime we implement a new system or put in a packet-related fix. We would just add the update to the main patch files for the expansion and the mini-patch files would inherit the update without needing to be touched.

Another possible (but less ideal) solution would be to only support something like 2 new client versions at a time. So, every 60 days (assuming that is the schedule for Steam updates), we would delete the client from 2 updates ago and add the latest one in its place. This way, people who already have the current client will not run into issues at all until the next update, and the latest/new version will be added and compatible ASAP so people getting the new download will be able to use it soon after the update happens. This solution would work, but is extra headache for players. It would mean players would have to download a new client at least every 60 days, and copy over their character files, maps, update their UI, etc. each time.

We could also possibly consider coding the packets that get sent out to be sent in the format used by the latest clients and then adding encodes to the older clients and removing them from the newer ones. This might make it easier to feed through the newer patch files, and would allow us to send all of the correct data from the server instead of having to fake it or put in hacks on the encodes and decodes for newer clients. It would be a pretty good amount of work and testing through. It would be good to do, but I am not sure that it would really help with the issue of having multiple patch versions to deal with.

No matter what the solution is; if we decide to start keeping up with the Steam updates, it would also mean all servers that want to allow the newer clients would have to update after each new client is added. Otherwise players running the latest client won't be able to play there, which would cause problems for all people new to EQEmu at that time.

Some of the goals for a solution are:
1. Minimal time investment required for devs to make new clients fully compatible.
2. Minimal work required to add new systems and packet-related fixes to previous clients.
3. Ability to continue supporting previous clients within reason.
4. Least amount of impact on Players.
5. Simplifying/automating the dev update process where possible.

Anyone that has ideas and is knowledgeable of the workings of the EQEmu patch files, feel free to chime in!