ArcCatalog error: Failed to connect the specified server. Server library could not be loaded

When trying to make a direct connect to ArcSDE on 64 bit Solaris, the following error occurred:
Failed to connect ot database. Server library could not be loaded.

There is an ESRI document on this:

Error:  Failed to connect the specified server. Server library could not be loaded

Article ID:	36510
Software:	ArcGIS - ArcEditor 9.2, 9.3, 9.3.1 ArcGIS - ArcInfo 9.2, 9.3, 9.3.1 ArcGIS - ArcView 9.2, 9.3, 9.3.1 ArcSDE 9.2, 9.3, 9.3.1
Platforms:	N/A

Error Message

When using direct connect with ArcSDE on a 64-bit operating system, the following error message is encountered:

"Failed to connect the specified server. Server library could not be loaded."

Cause

Because ArcGIS Desktop is a 32-bit operating system application, it cannot work with applications created with 64-bit operating system libraries.

Solution or Workaround

Uninstall the Oracle Net Client 64-bit version and install the Oracle Net Client 32-bit version on the 64-bit operating system. Also, ensure the Oracle Net Client's full version matches the Oracle database's full version (for example, 10.2.0.4).

Still it is very hard to work around the error!