When I was trying to create a signed JAR file that Firefox will allow to gain additional privileges, the biggest problem I faced was rooted at the fact that my certificate authority did not issue a certificate that entitled me to sign code. This simple fact was not clearly visible at the beginning, though.

After obtaining the certificate from my CA, I exported it with the corresponding private key in a PKCS12 format using OpenSSL to be imported into the key DB of NSS signtool as in "openssl pkcs12 -export -out cert+key.pkcs12 -in cert.pem -inkey key.pem && mkdir signtool_db && pk12util -i cert+key.pkcs12 -d signtool_db". The first attempt to sign a source directory with signtool as in "signtool -k Eus -d signtool_db -Z script.jar script/" returned the following error.

signtool: PROBLEM signing data (Certificate not approved for this operation)

Googling for the error message landed me on http://groups.google.com/group/mozilla.dev.tech.crypto/browse_thread/thr... whose important parts I quote below.

> using signtool -L -d. my cert has no asterisk before the name, I guess
> that is the problem. 

> Generating zigbert.sf file..
> signtool: PROBLEM signing data (Certificate not approved for this
> operation)

> Since it is a test certificate is not there a way to change it to be
> suitable for object signing as well?

There may be a way to force NSS to generate a signature anyway, but the
resultant signature would not be of value to others.

Let's say that you've listed your cert DB with certutil and found your
cert there to be named XXXXXXXX.  Then the command
   certutil -d DB -M -t "u,u,Pu" -n "XXXXXXXX"
(where DB is the name of the directory with your cert DB)
will set an override flag that MAY enable you to generate a signature
anyway, but the resultant signature will be of no value to anyone who
doesn't have your cert in their cert DB marked in that special way.

Since I didn't know about the existance of certificate extension, I thought I got the error because signtool didn't have my CA in its list of trusted CAs. So, I tried the certutil command and could sign the source directory.

However, opening the signed JAR file in Firefox at URL like "file:///some_directories/signed_file.jar!/file_with_script_requesting_privilege.html" threw this message: Signature Verification Error: the signature on index.html is invalid because the digital signature (*.RSA) file is not a valid signature of the signature instruction file (*.SF) like in .

Googling for the error message landed me on http://groups.google.com/group/mozilla.dev.tech.crypto/browse_thread/thr... whose important parts I quote below.

>> What software displayed that error message?

> I have the same error ... in the FireFox javascript: console.

If you use signtool to verify the signature on your jar file,
what does it report?

  signtool -v my.jar

Similarly, what does

  signtool -w my.jar

tell you?

I suggest that you do this:
- exit your browser
- copy the *.db files from your browser profile directory to some new
test directory
- run the above signtool commands with the -d option, telling signtool
the name of that test directory, e.g.
   signtool -d "mytestdirectory" -v my.jar

I tried to run the above commands against the signtool DB that I created earlier and everything went well. But, since at this point I knew that each Firefox profile has its own certificate DB, I tried to run the above commands against the Firefox profile DB as in "signtool -v script.jar -d ~/.mozilla/firefox/profile_dir/" and got the following result.

NOTE -- "script.jar" archive DID NOT PASS crypto verification.
  (reported reason: Certificate not approved for this operation)

entries shown below will have their digests checked only.

          status   path
    ------------   -------------------
        verified   index.html

NOTE -- "script.jar" archive DID NOT PASS crypto verification.

The reported reason, however, is much better than what is given in the Error Console. I think the one issued in the Error Console is suggesting that there is a bug in the signing tool because Firefox verifies that the generated signature doesn't match the file. Based on the reported reason, I went to look for certificate purpose and got what I wanted as I told about it here.

To conclude, when dealing with signed code in Firefox and you got message "Signature Verification Error: the signature on index.html is invalid because the digital signature (*.RSA) file is not a valid signature of the signature instruction file (*.SF)", you should try to execute "signtool -v YOUR_JAR_FILE -d FIREFOX_PROFILE_DIRECTORY" to see the real problem.