The format for all modern video codecs is not the kind of format where any specific piece of uncompressed input should always be encoded the same way, but more like a very restricted programming language that gives the encoder a lot of tools to compress the video, and which tools they use and how they use them are up to them.
I think it's neither the encoder nor the decoder that's specified but rather the encoding ie the bitstream format. At least for video streams building a competitive encoder is far more difficult than the corresponding decoder.
