Links may not function; however, this content may be relevant to outdated versions of the product.
Troubleshooting: Locks on multiplexer (Muxer) socket threads causing intermittent stalling of user nodes and non-responsive application server (WebLogic)
Symptom
With a Process Commander deployment running on the IBM AIX operating system and using the Oracle WebLogic application server, certain versions of the Oracle Service Bus and the WebLogic Server do not respond. User nodes managed by the WebLogic Server stall intermittently several times a day. The application server freezes and does not respond to any requests. The WebLogic administrative console is non-responsive.
This problem has been reported for the following Oracle components:
- Oracle Service Bus Versions 2.6.1RP to 10gR3
- Oracle WebLogic Server Versions 9.2 to 10.3
The problem is caused by locks on multiplexer (Muxer) socket threads. Correct this problem by applying patches that are available for Oracle WebLogic 9.2, 10.0, and 10.3 Maintenance Packs (MPs).
Explanation
The weblogic.socket.Muxer
module manages the server’s existing socket connections. Socket Reader Threads accept the incoming request from the Listen Thread Queue and put it on the Execute Thread Queue. First the Muxer determines which sockets have incoming requests waiting to be processed. Then it reads enough data to determine the protocol and dispatches the socket to an appropriate runtime layer based on the protocol. In the runtime layer, the Muxer determines which Execute Thread Queue should be used and delegates the request accordingly.
WebLogic 8.1 provided three socket reader threads by default. WebLogic 9 and later versions allocate 33 percent of server threads to be socket readers. While one socket reader polls, the others process requests.
Because of an application server defect, the socket Muxer threads are reported as stuck; they are all locked on the same object. The locks are transitory because they move from thread to thread: A thread gains an exclusive lock on an object to perform some action, after which it releases the lock to allow the next thread to lock the object. For complete information on WebLogic Muxers, including the two types (Java Muxer, Native Muxer), refer to the WebLogic Server Performance and Tuning guide that is cited as a reference.
Solution
To stabilize the application server, use Oracle Smart Update to download and apply the appropriate patch for your version of WebLogic:
WebLogic Application Server Version | Patch ID | Passcode |
---|---|---|
9.2 MP2 | WJD2 | GU1CW2AB |
10.0 MP1 | ITVL | K8RBHQQ2 |
10.3 | 9YT5 | I1DB5QSV |
Oracle WebLogic References
Oracle WebLogic Smart Upgrade Downloads
WebLogic Server Performance and Tuning: Tuning WebLogic Server, Tuning Network I/O, Tuning Muxers
BEA WebLogic Server 10.3 Tech Preview Documentation
BEA WebLogic Server 10.0 Documentation
BEA WebLogic Server 9.2 Documentation