• I have 2 sites on this server (dedicated server, fully controlled by me).
• Both sites are on Joomla 4, CB 2.8.2 & CB Subs 4.6.2, PHP 8.1
• Both sites only have PayPal and Stripe as payment options. 
• Both sites have Gateways with secondary “owners” (Going directly to other PayPal accounts I don’t have access to). The secondary gateways are only PayPal (not Stripe)
• Over the last couple of weeks, about a dozen IPN notifications failed from PayPal
• I was notified a few days ago that some subscription payments were not showing on the site
• Other IPNs which came before, between and after those that failed, went through OK.
• I get about a dozen IPNs a day, all going through fine, except for the aforementioned.
• As far as I can see, all the failed IPN included members who also pay to secondary gateways (meaning gateways with other owners). Obviously, the payments to different Gateways (and owners) are completely separate and independent transactions, the setup won't allow combining them. 
• Members that had a secondary gateway payment who’s IPN failed, also had their payment to the master gateway IPN fail, even as these were completely independent payments. Making it highly coincidental.
• This is making very few users have multiple IPN failures.
• 2 members, who were unaware their payment had gone through, but the IPN failed, when they couldn't access the site, they paid again, and the IPN failed again. Again, highly coincidental.
• All retries of the failed IPNs (16 retries for each IPN) failed. This is while other IPNs go through OK.
• I manually forced the failed IPNs in PayPal to retry, but all failed again. 
• I didn’t see any stepping on each other (the same second), but I’m not sure I would see this as the IPN’s failed and some don't come from my PayPal. Seems very unlikely though.
• To my knowledge, the site has never been down showing a 500 error
• There have been no issues with Stripe, but there are no secondary gateways with Stripe.
• I migrated the first site with less traffic to Joomla 4 on September 10. 
• I migrated the second site to Joomla 4 on October 4. 
• The first IPN fail was on October 1st, for the first site. I'm not aware of these IPN failures before the upgrades. 
• Today, there was another IPN fail (PayPal states Error 500) although the site has been up and there were successful IPNs and webhooks before and after (spaced by 20 minutes or so).

krileon wrote:

CBSubs should be logging those failed IPNs within CBSubs > Notifications as well as within CBSubs > History Log. Are there no failed notification entries and no errors logged for those failed IPNs? That should tell you exactly what the error is.

krileon wrote:

 If nothing is logged it's possible something installed is causing an error before CBSubs processes anything. In which case are there any errors in your server error log around the time of the failed IPNs?

mysite2.com:443 173.0.81.65 - - [07/Oct/2023:12:10:10 +0200] "POST /index.php?option=com_comprofiler&view=pluginclass&plugin=cbpaidsubscriptions&cbpgacctno=2&cbpbasket=3336&do=gw&result=notify&cbpid=cbp5faxxxxxxxxxxxxxxx979&format=raw HTTP/1.1" 200 5008 "-" "PayPal IPN ( https://www.paypal.com/ipn )"
mysite2.com:443 173.0.81.140 - - [07/Oct/2023:12:10:12 +0200] "POST /index.php?option=com_comprofiler&view=pluginclass&plugin=cbpaidsubscriptions&cbpgacctno=1&cbpbasket=6521&do=gw&result=notify&cbpid=cbp622xxxxxxxxxxxxxxx851&format=raw HTTP/1.1" 500 25086 "-" "PayPal IPN ( https://www.paypal.com/ipn )"
...
mysite2.com:443 173.0.81.65 - - [07/Oct/2023:12:12:15 +0200] "POST /index.php?option=com_comprofiler&view=pluginclass&plugin=cbpaidsubscriptions&cbpgacctno=1&cbpbasket=6521&do=gw&result=notify&cbpid=cbp622xxxxxxxxxxxxxxx851&format=raw HTTP/1.1" 500 25086 "-" "PayPal IPN ( https://www.paypal.com/ipn )"