...
Assuming that the installer application is located in the downloads directory and the particular version is 5.4.302:
C:\Downloads>QWS3270_Setup_5.4.302.exe DEFAULT_SSL_LIBRARY=OpenSSL DEFAULT_SSL_FIPS=1
This sets the default security SSL/Library to use OpenSSL for any session which specifies the Auto-Select option, and sets FIPS compliance to True.
C:\Downloads>QWS3270 Downloads>msiexe /i QWS3270_Setup_5.4.302.msi DEFAULT_SSL_LIBRARY=SChannel
This sets the default security SSL/Library to use SChannel for any session which specifies the Auto-Select option. The de facto default for FIPS (False) will used.
C:\Downloads>QWS3270_Setup_5.4.302.exe DEFAULT_SSL_FIPS=1
...