The first thing to do is to ignore all of the advice Cadence gives on setting the tolerances tighter than the default. I have used SpectreRF since before it was released, and I always start the default tolerances and they are sufficient almost all of the time. Most to the time when people complain about convergence, I strip off the tight tolerances and the convergence problems go away. And the simulation run much faster.

Second, keep an eye on the convergence norm. If it drops steadily until close to one (one represents convergence), but then stalls, it suggests that your tolerances are too tight.

Third, make sure your circuit is periodic. The easiest way to do this is to turn on strobing and strobe at the period of the fundamental frequency. If you then plot the strobed waveforms they should become constant valued. If they don't, if there is some noise or oscillation, then your circuit is not periodic and PSS will not converge.

Fourth, increase tstab. If you have plotted the strobed waveforms, set tstab to the time when they are close to being constant valued.

That is the process I use, and I very rarely have convergence problems with SpectreRF.

-Ken

I use the small-signal IP3 method described in http://www.designers-guide.org/Analysis/intercept-point.pdf and have never had a problem with accuracy.

-Ken

Why would the small-signal approach to IP₃ not be valid on class AB transmitters?

-Ken

Even order terms play no role in IP₃ as they generate responses that are at the wrong frequencies. The high order odd terms could play a role, but generally IP₃ is not used when they are significant. That is usually ACPR or EVM territory.

-Ken

A modulated carrier input to a cascade of two second-order nonlinearities generates a third-order intermodulation term which is typically denoted as a "second-order interaction" contribution to third-order distortion.

If the low frequency baseband terms of the first source of second-order distortion experience a filter function then positive and negative baseband frequencies will have a complex conjugate relationship. When the filtered baseband tones remix with the desired signal through the subsequent second-order nonlinearity, third-order intermod products are generated which have conjugate symmetry. These second-order interaction products can add in and out of phase with the primary third-order products producing the classic asymmetric third-order intermodulation terms which are indicative of the so called "long term memory" effects in RF power amplifiers.

I can post a more elaborate expanation; however, there is a good reference on the topic with clear derivations.

K. A. Remley, D. F. Williams, D. M. M.-P. Schreurs, and J. Wood, "Simplifying and Interpreting Two-Tone Measurements," IEEE Trans. Microwave Theory and Techn., vol. 52, no. 11, 2004, pp. 2576-2584.

Sections III.B and III.C discuss the generation of third-order products from second order interaction with baseband and second harmonic products.