Answer
Import the self-signed certificate as a Trusted Root CA on the client machine — browse to the Application Server's access point URL in a browser that lets you view and install the presented certificate, add it under Trusted Root Certification Authorities, then clear the browser cache before retrying the connection. If that still doesn't resolve it, the self-signed certificate itself may be malformed — regenerate it through the installer, making sure the certificate details (like the hostname/Common Name) are entered correctly.