Finding Opcodes

[KLS]

We use a protocol similar to the one used in eq2 I believe though it really isn't documented anywhere outside of the code. It's kind of difficult to put together via the code because of the layers of abstraction on abstraction. It would just be a matter of decoding the packets further to get them to load from something like WireShark.

Usually the opcodes change from exe version to version though some don't; sony does seem to like to change a lot of the ingame ones every patch; probably to hinder stuff like showeq.

Usually if the client sends a packet and expects a direct response packet for it they will use the same opcode; example would be pickpocket or click object.

It's a shame nothing ever came of openeq and we still have to deal with this. =/

[John Adams]

Heh KLS... this sounds hauntingly familiar.

[KLS]

Sir, I thought I was leaving you alone with this!

Anyway basically the packets are encrypted or compressed, there's a flag on the session request for the stream that indicates which it will be; I believe it can be both as well. The server generates a key to send to the client to decode the data as well.

You have larger packets that make up the protocol, things like session requests, whole packets and combined packets etc. Contained within can be the smaller packets, encoded with the key sent from the server etc.

I think we send the key always as 11223344 but obviously were you to do this from live or something you would need to retrieve the actual one to decode the data into readable packets.

in common/EQPacket.cpp there are several functions that deal with compression and encoding of the data as well.

[trevius]

Thanks for the info KLS, it seems like it has been a while since anyone has considered taking on opcodes and I wouldn't mind giving it a try as long as I can find a viable option for it.

Here are some code sections from the source common directory that just skimming through caught my eye as being related to helping find opcodes:

packetfile.h - Guessing this holds the string you were mentioning that the server sends the client, maybe?

#ifndef PACKET_FILE_H
#define PACKET_FILE_H

#include "../common/types.h"
#include <stdio.h>
#include <time.h>
//#include <zlib.h>

//constants used in the packet file header
#define PACKET_FILE_MAGIC 0x93a7b6f6
#define OLD_PACKET_FILE_MAGIC 0x93a7b6f7

#define PACKET_FILE_CURRENT_VERSION 1

#pragma pack(1)
//old structs from when I forgot to put the version number in
struct OldPacketFileHeader {
	uint32 packet_file_magic;
	uint32 packet_file_stamp;
};
struct OldPacketFileSection {
	uint16 opcode;
	uint32 len;
};

struct PacketFileHeader {
	uint32 packet_file_magic;
	uint16 packet_file_version;
	uint32 packet_file_stamp;
};

struct PacketFileSection {
	uint16 opcode;
	uint8 flags;	//mainly for client->server, but others could be added
	uint32 tv_sec;
	uint16 tv_msec;
	uint32 len;
};
#pragma pack()

#define TO_SERVER_FLAG 0x01
#define SetToClient(pfs) pfs.flags = pfs.flags&~TO_SERVER_FLAG
#define SetToServer(pfs) pfs.flags = pfs.flags|TO_SERVER_FLAG
#define IsToClient(pfs) (pfs.flags&TO_SERVER_FLAG == 0)
#define IsToServer(pfs) (pfs.flags&TO_SERVER_FLAG != 0)


class PacketFileWriter {
public:
	PacketFileWriter(bool force_flush);
	~PacketFileWriter();
	
	bool OpenFile(const char *name);
	void CloseFile();
	
	void WritePacket(uint16 eq_op, uint32 packlen, const unsigned char *packet, bool to_server, const struct timeval &tv);
	
	static bool SetPacketStamp(const char *file, uint32 stamp);
	
protected:
	bool _WriteBlock(uint16 eq_op, const void *d, uint16 len, bool to_server, const struct timeval &tv);
	
	//gzFile out;
	FILE *out;
	bool force_flush;
};


class PacketFileReader {
public:
	PacketFileReader();
	
	virtual bool OpenFile(const char *name) = 0;
	virtual void CloseFile() = 0;
	virtual bool ResetFile() = 0;	//aka rewind
	
	virtual bool ReadPacket(uint16 &eq_op, uint32 &packlen, unsigned char *packet, bool &to_server, struct timeval &tv) = 0;
	
	time_t GetStamp() { return(time_t(packet_file_stamp)); }
	
	//factory method to open the right packet file.
	static PacketFileReader *OpenPacketFile(const char *name);
	
protected:
	
	uint32 packet_file_stamp;
};

class OldPacketFileReader : public PacketFileReader {
public:
	OldPacketFileReader();
	virtual ~OldPacketFileReader();
	
	bool OpenFile(const char *name);
	void CloseFile();
	bool ResetFile();	//aka rewind
	
	bool ReadPacket(uint16 &eq_op, uint32 &packlen, unsigned char *packet, bool &to_server, struct timeval &tv);
	
	time_t GetStamp() { return(time_t(packet_file_stamp)); }
	
protected:
	
	//gzFile in;
	FILE *in;	
};

class NewPacketFileReader: public PacketFileReader {
public:
	NewPacketFileReader();
	virtual ~NewPacketFileReader();
	
	bool OpenFile(const char *name);
	void CloseFile();
	bool ResetFile();	//aka rewind
	
	bool ReadPacket(uint16 &eq_op, uint32 &packlen, unsigned char *packet, bool &to_server, struct timeval &tv);
	
	time_t GetStamp() { return(time_t(packet_file_stamp)); }
	
protected:
	
	//gzFile in;
	FILE *in;	
};


#endif

EQPacket.h - Adding this here for reference

#ifndef _EQPACKET_H
#define _EQPACKET_H

#include "BasePacket.h"
#include "EQStreamType.h"
#include "op_codes.h"

#ifdef STATIC_OPCODE
	typedef unsigned short EmuOpcode;
	static const EmuOpcode OP_Unknown = 0;
#else
#include "emu_opcodes.h"
#endif

using namespace std;

class EQStream;
class EQStreamPair;

class EQPacket : public BasePacket {
	friend class EQStream;
public:
	virtual ~EQPacket() {}
	
	uint32 Size() const { return size+2; }
	
	virtual void build_raw_header_dump(char *buffer, uint16 seq=0xffff) const;
	virtual void build_header_dump(char *buffer) const;
	virtual void DumpRawHeader(uint16 seq=0xffff, FILE *to = stdout) const;
	virtual void DumpRawHeaderNoTime(uint16 seq=0xffff, FILE *to = stdout) const;

	void SetOpcode(EmuOpcode op) { emu_opcode = op; }
	const EmuOpcode GetOpcode() const { return(emu_opcode); }
//	const char *GetOpcodeName() const;
	
protected:
	//this is just a cache so we dont look it up several times on Get()
	//and it is mutable so we can store the cached copy even on a const object
	EmuOpcode emu_opcode;

	EQPacket(EmuOpcode opcode, const unsigned char *buf, const uint32 len);
//	EQPacket(const EQPacket &p) { }
	EQPacket() { emu_opcode=OP_Unknown; pBuffer=NULL; size=0; }

};

class EQRawApplicationPacket;

class EQProtocolPacket : public BasePacket {
	friend class EQStream;
	friend class EQStreamPair;
public:
	EQProtocolPacket(uint16 op, const unsigned char *buf, uint32 len) : BasePacket(buf,len), opcode(op) { } 
//	EQProtocolPacket(const unsigned char *buf, uint32 len);
	bool combine(const EQProtocolPacket *rhs);
	uint32 serialize (unsigned char *dest) const;
	EQProtocolPacket *Copy() { return new EQProtocolPacket(opcode,pBuffer,size); }
	EQRawApplicationPacket *MakeAppPacket() const;
	
	
	virtual void build_raw_header_dump(char *buffer, uint16 seq=0xffff) const;
	virtual void build_header_dump(char *buffer) const;
	virtual void DumpRawHeader(uint16 seq=0xffff, FILE *to = stdout) const;
	virtual void DumpRawHeaderNoTime(uint16 seq=0xffff, FILE *to = stdout) const;
	
protected:
	
	static bool ValidateCRC(const unsigned char *buffer, int length, uint32 Key);
	static uint32 Decompress(const unsigned char *buffer, const uint32 length, unsigned char *newbuf, uint32 newbufsize);
	static uint32 Compress(const unsigned char *buffer, const uint32 length, unsigned char *newbuf, uint32 newbufsize);
	static void ChatDecode(unsigned char *buffer, int size, int DecodeKey);
	static void ChatEncode(unsigned char *buffer, int size, int EncodeKey);
	
	uint16 GetRawOpcode() const { return(opcode); }
	
	uint32 Size() const { return size+2; }
	
	//the actual raw EQ opcode
	uint16 opcode;
};

class EQApplicationPacket : public EQPacket {
//	friend class EQProtocolPacket;
	friend class EQStream;
public:
	EQApplicationPacket() : EQPacket(OP_Unknown,NULL,0) { app_opcode_size=2; }
	EQApplicationPacket(const EmuOpcode op) : EQPacket(op,NULL,0) { app_opcode_size=2; }
	EQApplicationPacket(const EmuOpcode op, const uint32 len) : EQPacket(op,NULL,len) { app_opcode_size=2; }
	EQApplicationPacket(const EmuOpcode op, const unsigned char *buf, const uint32 len) : EQPacket(op,buf,len) { app_opcode_size=2; }
	bool combine(const EQApplicationPacket *rhs);
	uint32 serialize (uint16 opcode, unsigned char *dest) const;
	uint32 Size() const { return size+app_opcode_size; }
	
	virtual EQApplicationPacket *Copy() const;
	
	virtual void build_raw_header_dump(char *buffer, uint16 seq=0xffff) const;
	virtual void build_header_dump(char *buffer) const;
	virtual void DumpRawHeader(uint16 seq=0xffff, FILE *to = stdout) const;
	virtual void DumpRawHeaderNoTime(uint16 seq=0xffff, FILE *to = stdout) const;
	
protected:
	int8 app_opcode_size;

private:

	EQApplicationPacket(const EQApplicationPacket &p) : EQPacket(p.emu_opcode, p.pBuffer, p.size) { app_opcode_size = p.app_opcode_size; }

};

class EQRawApplicationPacket : public EQApplicationPacket {
	friend class EQStream;
public:
	EQRawApplicationPacket(uint16 opcode, const unsigned char *buf, const uint32 len);
	uint16 GetRawOpcode() const { return(opcode); }
	
	virtual void build_raw_header_dump(char *buffer, uint16 seq=0xffff) const;
	virtual void build_header_dump(char *buffer) const;
	virtual void DumpRawHeader(uint16 seq=0xffff, FILE *to = stdout) const;
	virtual void DumpRawHeaderNoTime(uint16 seq=0xffff, FILE *to = stdout) const;

protected:
	
	//the actual raw EQ opcode
	uint16 opcode;
	
	EQRawApplicationPacket(const unsigned char *buf, const uint32 len);
};

extern void DumpPacket(const EQApplicationPacket* app, bool iShowInfo = false);


#endif

StructStrategy.cpp - Maybe this is where the Opcodes get encrypted/decrypted?

#include "debug.h"
#include "StructStrategy.h"
#include "logsys.h"
#include "EQStream.h"
#include <map>


//note: all encoders and decoders must be valid functions.
//so if you specify set_defaults=false
StructStrategy::StructStrategy() {
	int r;
	for(r = 0; r < _maxEmuOpcode; r++) {
		encoders[r] = PassEncoder;
		decoders[r] = PassDecoder;
	}
}

void StructStrategy::Encode(EQApplicationPacket **p, EQStream *dest, bool ack_req) const {
	EmuOpcode op = (*p)->GetOpcode();
	Encoder proc = encoders[op];
	proc(p, dest, ack_req);
}

void StructStrategy::Decode(EQApplicationPacket *p) const {
	EmuOpcode op = p->GetOpcode();
	Decoder proc = decoders[op];
	proc(p);
}

	
void StructStrategy::ErrorEncoder(EQApplicationPacket **in_p, EQStream *dest, bool ack_req) {
	EQApplicationPacket *p = *in_p;
	*in_p = NULL;
	
	_log(NET__STRUCTS, "Error encoding opcode %s: no encoder provided. Dropping.", OpcodeManager::EmuToName(p->GetOpcode()));
	
	delete p;
}

void StructStrategy::ErrorDecoder(EQApplicationPacket *p) {
	_log(NET__STRUCTS, "Error decoding opcode %s: no decoder provided. Invalidating.", OpcodeManager::EmuToName(p->GetOpcode()));
	p->SetOpcode(OP_Unknown);
}

void StructStrategy::PassEncoder(EQApplicationPacket **p, EQStream *dest, bool ack_req) {
	dest->FastQueuePacket(p, ack_req);
}

void StructStrategy::PassDecoder(EQApplicationPacket *p) {
	//do nothing since we decode in place
}




//effectively a singleton, but I decided to do it this way for no apparent reason.
namespace StructStrategyFactory {
	
	static map<EmuOpcode, const StructStrategy *> strategies;
	
	void RegisterPatch(EmuOpcode first_opcode, const StructStrategy *structs) {
		strategies[first_opcode] = structs;
	}
	
	const StructStrategy *FindPatch(EmuOpcode first_opcode) {
		map<EmuOpcode, const StructStrategy *>::const_iterator res;
		res = strategies.find(first_opcode);
		if(res == strategies.end())
			return(NULL);
		return(res->second);
	}
	
};

/utils/throwpackets.pl - Not sure what this script does, but it sounded like it could be something Opcode utility related.

#copy this file into your server's dir
#include this file with: require "throwpackets.pl";
# and add:
# command_add("throwfile", "[opcode name] [filename] - Send a file's hex contents as a packet", 250);
#to your commands_init to enable this


sub throwfile {
	my $op = shift;
	my $file = shift;
	my $p = FileToPacket($op, $file);
	if(!$p) {
		$client->Message(13, "Unable to read file or parse contents.");
		return;
	}
	$p->SendTo($client);
	$client->Message(0, "Sent.");
}

sub HexToPacket {
	my $op = shift;
	my $hex = shift;
	my @lines = split(/\r?\n/, $hex);
	my $body = "";
	my @pieces = ();
	foreach my $l (@lines) {
		if($l =~ /[0-9a-fA-Fx]+:\s*(.*)\s+\|/) {
			$l = $1;
		}
		$l =~ s/\s+-\s+/ /g;
		$body .= $l;
	}
	foreach my $p (split(/\s+/, $body)) {
		push(@pieces, "0x$p");
	}
	my $p = new PerlPacket($op);
	$p->FromArray(\@pieces, $#pieces+1);
	return($p);
}

sub FileToPacket {
	my $op = shift;
	my $file = shift;
	my $c = "";
	open(F, "<$file") || return(undef);
	while(<F>) {
		$c .= $_;
	}
	close(F);
	return(HexToPacket($op, $c));
}

utils/asmtools/stringids_to_ida.pl - Guessing by the name of this and other .pl files in this directory that these are scripts to help pull info from the IDA Pro files created by examining the game file. Now, if I could only figure out how to use these and if they are updated properly to work with Titanium or maybe later expansions.

#!/usr/bin/perl

#reads the output of locate_stringids.pl from stdin
#produces a .IDC file on stdout which we can feed to IDA

print "#include \"idc.idc\"\n\nstatic main() {\n";

while(<>) {
	s/\r?\n//g;
	next unless(/^([0-9a-fA-F]+) Sends (.*)/);
	next if(hex($1) == 0);
	my $off = hex($1);
	my $str = substr($2, 0, 200);
	$str =~ s/"/\\"/g;
	printf("\tMakeComm($off, \"$str\");\n");
}

print "}\n\n";

/utils/asmtools/opcodes_to_ida.pl - Same info as noted above

#!/usr/bin/perl

print "#include \"idc.idc\"\n\nstatic main() {\n";

print "\tauto id;\n";
print "\tDelEnum(GetEnum(\"EQOpcode\"));\n";
print "\tid = AddEnum(GetEnumQty(), \"EQOpcode\", 0);\n";

while(<>) {
	next unless(/^(OP_[^= \t]+)=(0x[0-9a-fA-F]+)/);
	next if($2 eq "0x0000" || hex($2) == 0);
	printf("\tAddConstEx(id, \"$1\", $2, -1);\n");
}

print "}\n\n";

/common/logtypes.h - I noticed that this file contains a bunch of settings for log files. I don't know if this is used anymore, or if this was the old way of logging that has an option in the variables for setting log levels or something.

#ifndef LOG_CATEGORY
#define LOG_CATEGORY(name)
#endif
#ifndef LOG_TYPE
#define LOG_TYPE(cat, type, default_value)
#endif
#ifndef ENABLED
#define ENABLED true
#endif
#ifndef DISABLED
#define DISABLED false
#endif




LOG_CATEGORY( CHAT )
LOG_TYPE( CHAT, SAY, DISABLED )
LOG_TYPE( CHAT, EMOTE, DISABLED )
LOG_TYPE( CHAT, OOC, DISABLED )
LOG_TYPE( CHAT, GROUP, DISABLED )
LOG_TYPE( CHAT, GUILD, DISABLED )

LOG_CATEGORY( SPAWNS )
LOG_TYPE( SPAWNS, MAIN, DISABLED )
LOG_TYPE( SPAWNS, CONDITIONS, DISABLED )
LOG_TYPE( SPAWNS, LIMITS, DISABLED )

LOG_CATEGORY( AI )
LOG_TYPE( AI, ERROR, ENABLED )
LOG_TYPE( AI, WAYPOINTS, DISABLED )
LOG_TYPE( AI, BUFFS, DISABLED )
LOG_TYPE( AI, SPELLS, DISABLED )

LOG_CATEGORY( QUESTS )
LOG_TYPE( QUESTS, PATHING, DISABLED )

LOG_CATEGORY( SPELLS )
LOG_TYPE( SPELLS, LOAD, DISABLED )
LOG_TYPE( SPELLS, LOAD_ERR, DISABLED )
LOG_TYPE( SPELLS, CASTING_ERR, DISABLED )
LOG_TYPE( SPELLS, CASTING, DISABLED )
LOG_TYPE( SPELLS, EFFECT_VALUES, DISABLED )
LOG_TYPE( SPELLS, RESISTS, DISABLED )
LOG_TYPE( SPELLS, STACKING, DISABLED )
LOG_TYPE( SPELLS, BARDS, DISABLED )
LOG_TYPE( SPELLS, BUFFS, DISABLED )
LOG_TYPE( SPELLS, PROCS, DISABLED )
LOG_TYPE( SPELLS, MODIFIERS, DISABLED )

LOG_CATEGORY( FACTION )

LOG_CATEGORY( ZONE )
LOG_TYPE( ZONE, GROUND_SPAWNS, DISABLED )
LOG_TYPE( ZONE, INIT, ENABLED )
LOG_TYPE( ZONE, INIT_ERR, ENABLED )
LOG_TYPE( ZONE, WORLD, ENABLED )
LOG_TYPE( ZONE, WORLD_ERR, ENABLED )
LOG_TYPE( ZONE, WORLD_TRACE, DISABLED )

LOG_CATEGORY( TRADING )
LOG_TYPE( TRADING, ERROR, ENABLED )
LOG_TYPE( TRADING, CLIENT, DISABLED )
LOG_TYPE( TRADING, NPC, DISABLED )
LOG_TYPE( TRADING, HOLDER, DISABLED )

LOG_CATEGORY( INVENTORY )
LOG_TYPE( INVENTORY, ERROR, ENABLED )
LOG_TYPE( INVENTORY, SLOTS, ENABLED )

LOG_CATEGORY( TRADESKILLS )
LOG_TYPE( TRADESKILLS, IN, DISABLED )
LOG_TYPE( TRADESKILLS, OUT, DISABLED )
LOG_TYPE( TRADESKILLS, SQL, DISABLED )
LOG_TYPE( TRADESKILLS, TRACE, DISABLED )

LOG_CATEGORY( TRIBUTE )
LOG_TYPE( TRIBUTE, ERROR, DISABLED )
LOG_TYPE( TRIBUTE, IN, DISABLED )
LOG_TYPE( TRIBUTE, OUT, DISABLED )

LOG_CATEGORY( AA )
LOG_TYPE( AA, ERROR, ENABLED )
LOG_TYPE( AA, MESSAGE, DISABLED )
LOG_TYPE( AA, IN, DISABLED )
LOG_TYPE( AA, OUT, DISABLED )


LOG_CATEGORY( DOORS )
LOG_TYPE( DOORS, INFO, DISABLED )

LOG_CATEGORY( PETS )
LOG_TYPE( PETS, AGGRO, DISABLED )

LOG_CATEGORY( COMBAT )
LOG_TYPE( COMBAT, ATTACKS, DISABLED )
LOG_TYPE( COMBAT, TOHIT, DISABLED )
LOG_TYPE( COMBAT, MISSES, DISABLED )
LOG_TYPE( COMBAT, DAMAGE, DISABLED )
LOG_TYPE( COMBAT, HITS, DISABLED )
LOG_TYPE( COMBAT, RANGED, DISABLED )
LOG_TYPE( COMBAT, SPECIAL_ATTACKS, DISABLED )
LOG_TYPE( COMBAT, PROCS, DISABLED )

LOG_CATEGORY( GUILDS )
LOG_TYPE( GUILDS, ERROR, ENABLED )
LOG_TYPE( GUILDS, ACTIONS, ENABLED )
LOG_TYPE( GUILDS, DB, DISABLED )
LOG_TYPE( GUILDS, PERMISSIONS, DISABLED )
LOG_TYPE( GUILDS, REFRESH, DISABLED )	//inter-zone refresh comm
LOG_TYPE( GUILDS, IN_PACKETS, DISABLED )
LOG_TYPE( GUILDS, OUT_PACKETS, DISABLED )
LOG_TYPE( GUILDS, IN_PACKET_TRACE, DISABLED )	//hex dumps
LOG_TYPE( GUILDS, OUT_PACKET_TRACE, DISABLED )	//hex dumps

LOG_CATEGORY( CLIENT )
LOG_TYPE( CLIENT, ERROR, ENABLED )
LOG_TYPE( CLIENT, DUELING, DISABLED )
LOG_TYPE( CLIENT, SPELLS, DISABLED )
LOG_TYPE( CLIENT, NET_ERR, ENABLED )
LOG_TYPE( CLIENT, NET_IN_TRACE, DISABLED )

LOG_CATEGORY( SKILLS )
LOG_TYPE( SKILLS, GAIN,  DISABLED )

LOG_CATEGORY( RULES )
LOG_TYPE( RULES, ERROR,  ENABLED )
LOG_TYPE( RULES, CHANGE, ENABLED )

LOG_CATEGORY( NET )
LOG_TYPE( NET, WORLD, ENABLED )
LOG_TYPE( NET, OPCODES, ENABLED )
LOG_TYPE( NET, IDENTIFY, ENABLED )
LOG_TYPE( NET, IDENT_TRACE, ENABLED )
LOG_TYPE( NET, STRUCTS, ENABLED )
LOG_TYPE( NET, STRUCT_HEX, ENABLED )
LOG_TYPE( NET, ERROR, ENABLED )
LOG_TYPE( NET, DEBUG, DISABLED )
LOG_TYPE( NET, APP_TRACE, DISABLED )
LOG_TYPE( NET, APP_CREATE, DISABLED )
LOG_TYPE( NET, APP_CREATE_HEX, DISABLED )
LOG_TYPE( NET, NET_TRACE, DISABLED )
LOG_TYPE( NET, NET_COMBINE, DISABLED )
LOG_TYPE( NET, FRAGMENT, DISABLED )
LOG_TYPE( NET, FRAGMENT_HEX, DISABLED )
LOG_TYPE( NET, NET_CREATE, DISABLED )
LOG_TYPE( NET, NET_CREATE_HEX, DISABLED )
LOG_TYPE( NET, NET_ACKS, DISABLED )
LOG_TYPE( NET, RATES, DISABLED )

LOG_CATEGORY( DATABASE )

LOG_CATEGORY( COMMON )
LOG_TYPE( COMMON, ERROR, ENABLED )
LOG_TYPE( COMMON, THREADS, ENABLED )

LOG_CATEGORY( LAUNCHER )
LOG_TYPE( LAUNCHER, ERROR, ENABLED )
LOG_TYPE( LAUNCHER, INIT, ENABLED )
LOG_TYPE( LAUNCHER, STATUS, ENABLED )
LOG_TYPE( LAUNCHER, NET, ENABLED )
LOG_TYPE( LAUNCHER, WORLD, ENABLED )

LOG_CATEGORY( WORLD )
LOG_TYPE( WORLD, CONFIG, ENABLED )
LOG_TYPE( WORLD, INIT, ENABLED )
LOG_TYPE( WORLD, INIT_ERR, ENABLED )
LOG_TYPE( WORLD, CLIENT, ENABLED )
LOG_TYPE( WORLD, ZONE, ENABLED )
LOG_TYPE( WORLD, LS, ENABLED )
LOG_TYPE( WORLD, CLIENT_ERR, ENABLED )
LOG_TYPE( WORLD, ZONE_ERR, ENABLED )
LOG_TYPE( WORLD, LS_ERR, ENABLED )
LOG_TYPE( WORLD, SHUTDOWN, ENABLED )
LOG_TYPE( WORLD, CLIENTLIST, DISABLED )
LOG_TYPE( WORLD, CLIENTLIST_ERR, ENABLED )
LOG_TYPE( WORLD, ZONELIST, ENABLED )
LOG_TYPE( WORLD, ZONELIST_ERR, ENABLED )
LOG_TYPE( WORLD, CLIENT_TRACE, DISABLED )
LOG_TYPE( WORLD, ZONE_TRACE, DISABLED )
LOG_TYPE( WORLD, LS_TRACE, DISABLED )
LOG_TYPE( WORLD, CONSOLE, ENABLED )
LOG_TYPE( WORLD, HTTP, ENABLED )
LOG_TYPE( WORLD, HTTP_ERR, ENABLED )
LOG_TYPE( WORLD, PERL, ENABLED )
LOG_TYPE( WORLD, PERL_ERR, ENABLED )
LOG_TYPE( WORLD, EQW, ENABLED )
LOG_TYPE( WORLD, LAUNCH, ENABLED )
LOG_TYPE( WORLD, LAUNCH_ERR, ENABLED )
LOG_TYPE( WORLD, LAUNCH_TRACE, ENABLED )

#undef LOG_TYPE
#undef LOG_CATEGORY

I am curious if there is a way to have the emu itself just log all opcodes instead of having to sniff them. Since it already encodes/decodes the opcodes and seems to already show some opcodes in the logs that are unknown, maybe all opcode information could get sent to one log file to help find them. Then you could just run a tail (in linux - "tail -f /home/server/logs/opcodes.log") to watch the opcodes come in real-time. If you are the only person on the server, it should be pretty easy to figure out what opcodes do what by just watching which ones come in when you do something. So, if you type /rewind, you should see opcode 0x4CFA come in right at the same time and you will know that those 2 are related. Then, it is just a matter of doing things to verify which opcode is for what one at a time. You will probably want to do some of them multiple times to make sure you have the right one.

It would also be nice if that was working and you could somehow filter out all of the known opcodes, so only ones you are looking for (unknowns) will show up to make finding them much easier and quicker. Then, once we were able to get enough opcodes to get a client logged all of the way in, the rest would probably be pretty easy.

[trevius]

Hmm, I may have answered my own question about using the logs to find opcodes. To watch them as they come in, simply open a terminal window in Linux and type:

cd /home/eqemu/server/logs
tail -f *.* | grep OpCode

You could probably add more stuff to grep for, but this seems to be a start at least lol. I already see some opcodes coming from my dozens of players that are unknowns. Now, if I can get a test server setup so that it is only me that is on, I can start testing and see if my idea works for defining opcodes. There are too many people on my server to use the system for testing without giving them all the boot for a while, so I will have to get a second one running.

Unfortunately my other PC is windows only, and I don't know how to do a tail in windows so I can grep for OpCodes only. So, I may have to build my other PC as dual boot to Debian Linux. Unless I can figure out how to run another server from the same PC I run my main server on. So, I will have 1 that only I can use. Maybe if I make another directory and set the config to use different zone ports... I dunno, but I don't think it would talk right on port 9000 with both server sessions wanting to use the same port to communicate with the login server.

[trevius]

Looks like I might be able to do this on my server even with players on it after-all. This tail shows the name of the char that sent the opcode, so if I am testing, I will just look for my own.

It has already found a few unknown opcodes from my players and these seem to be somewhat common.

OP_Unknowns:
0x6a5f
0x45ff
0x7085
0x3b21
0x1241

I will do some testing tonight to see if I can force it to produce more opcodes and hopefully get some of the unknowns defined. I will let you all know of any progress I make. If this actually works, it is far easier than I could have ever expected lol. Maybe I was thinking too hard, but I imagine that if it was this easy, someone else would have been using this method already...

[AndMetal]

I took a quick peek into the source, and it looks like this is where those OpCode errors you found are spit out:

zone/client_packet.cpp:

  375 	case CLIENT_CONNECTED: {
  376 		ClientPacketProc p;
  377 		p = ConnectedOpcodes[opcode];
  378 		if(p == NULL) {
  379 			char buffer[64];
  380 			app->build_header_dump(buffer);
  381 			mlog(CLIENT__NET_ERR, "Unhandled incoming opcode: %s", buffer);
  382 			if(app->size<1000)
  383 				DumpPacket(app->pBuffer, app->size);
  384 			else{
  385 				cout << "Dump limited to 1000 characters:\n";
  386 				DumpPacket(app->pBuffer, 1000);
  387 			}
  388 			break;
  389 		}
  390 
  391 		//call the processing routine
  392 		(this->*p)(app);
  393 		break;
  394 	}

You could relatively easily put some code in there to spit out the OpCode info to the terminal (cerr is used elsewhere in the source). Then again, looking just above the above code, someone already has:

  337 	#if EQDEBUG >= 9
  338 		cout << "Received 0x" << hex << setw(4) << setfill('0') << opcode << ", size=" << dec << app->size << endl;
  339 	#endif
  340 
  341 	#ifdef SOLAR
  342 		if(0 && opcode != OP_ClientUpdate)
  343 		{
  344 			LogFile->write(EQEMuLog::Debug,"HandlePacket() OPCODE debug enabled client %s", GetName());
  345 			cerr << "OPCODE: " << hex << setw(4) << setfill('0') << opcode << dec << ", size: " << app->size << endl;
  346 			DumpPacket(app);
  347 		}
  348 	#endif

So, you could set EQDEBUG=9 to get console output of what looks to be all of the OpCodes. This is done in the zone makefile before compiling:

   15 DFLAGS=-DEQDEBUG=5 -DCATCH_CRASH -DNO_PIDLOG -DSHAREMEM -DSPELL_EFFECT_SPAM -DFIELD_ITEMS -DCOMBINED -DAPP_OPCODE_SIZE=2 -Di386

If this works, I might put up a VM server to use locally to dig out some OpCodes. It sure beats packet sniffing + creating/fixing a program to decode it all Then, we'll just need to figure out the packet structures if we don't have them already.