[jira] [Created] (COCOON-2340) XMLByteStreamCompiler in not thread-safe

[Ivan Lagunov (JIRA)]

Ivan Lagunov created COCOON-2340:
------------------------------------

Summary: XMLByteStreamCompiler in not thread-safe
Key: COCOON-2340
URL: https://issues.apache.org/jira/browse/COCOON-2340
Project: Cocoon
Issue Type: Bug
Components: * Cocoon Core
Affects Versions: 2.2
Reporter: Ivan Lagunov


I'm periodically getting the following exceptions in different places:

Caused by: java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at org.apache.cocoon.components.sax.XMLByteStreamCompiler.getSAXFragment(XMLByteStreamCompiler.java:61)
at org.apache.cocoon.components.pipeline.impl.ExpiresCachingProcessingPipeline.processXMLPipeline(ExpiresCachingProcessingPipeline.java:152)

After debugging I've found the root cause. It happens due to multi-threading when one thread allocates newbuf array, then another thread increments bufCount in write method and then the first thread fails on System.arraycopy invocation.

I suggest making getSAXFragment and write methods synchronized. It should resolve the issue. If it helps, I'll provide a patch later.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[Ivan Lagunov (JIRA)]

[ https://issues.apache.org/jira/browse/COCOON-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13833877#comment-13833877 ]

Ivan Lagunov commented on COCOON-2340:
--------------------------------------

Synchronization helped to avoid the Exception although now the resource seems to be loaded partially. So the root cause is on the upper level. Perhaps, all writes to this.buf must be done before getSAXFragment invocation because all the data written to this.buf afterwards seems to be ignored.

Post by Ivan Lagunov (JIRA)
XMLByteStreamCompiler in not thread-safe
----------------------------------------
Key: COCOON-2340
URL: https://issues.apache.org/jira/browse/COCOON-2340
Project: Cocoon
Issue Type: Bug
Components: * Cocoon Core
Affects Versions: 2.2
Reporter: Ivan Lagunov
Caused by: java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at org.apache.cocoon.components.sax.XMLByteStreamCompiler.getSAXFragment(XMLByteStreamCompiler.java:61)
at org.apache.cocoon.components.pipeline.impl.ExpiresCachingProcessingPipeline.processXMLPipeline(ExpiresCachingProcessingPipeline.java:152)
After debugging I've found the root cause. It happens due to multi-threading when one thread allocates newbuf array, then another thread increments bufCount in write method and then the first thread fails on System.arraycopy invocation.
I suggest making getSAXFragment and write methods synchronized. It should resolve the issue. If it helps, I'll provide a patch later.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

[Francesco Chicchiriccò (JIRA)]

[ https://issues.apache.org/jira/browse/COCOON-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834586#comment-13834586 ]

Francesco Chicchiriccò commented on COCOON-2340:
------------------------------------------------

Hi,
I am not very familiar with 2.2, but if you can provide a patch, I can apply it to [trunk|http://svn.apache.org/repos/asf/cocoon/trunk/] and re-deploy the SNAPSHOT Maven artifacts containing your fix.

Post by Ivan Lagunov (JIRA)
XMLByteStreamCompiler in not thread-safe
----------------------------------------
Key: COCOON-2340
URL: https://issues.apache.org/jira/browse/COCOON-2340
Project: Cocoon
Issue Type: Bug
Components: * Cocoon Core
Affects Versions: 2.2
Reporter: Ivan Lagunov
Caused by: java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at org.apache.cocoon.components.sax.XMLByteStreamCompiler.getSAXFragment(XMLByteStreamCompiler.java:61)
at org.apache.cocoon.components.pipeline.impl.ExpiresCachingProcessingPipeline.processXMLPipeline(ExpiresCachingProcessingPipeline.java:152)
After debugging I've found the root cause. It happens due to multi-threading when one thread allocates newbuf array, then another thread increments bufCount in write method and then the first thread fails on System.arraycopy invocation.
I suggest making getSAXFragment and write methods synchronized. It should resolve the issue. If it helps, I'll provide a patch later.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

[Javier Puerto (JIRA)]

[ https://issues.apache.org/jira/browse/COCOON-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834654#comment-13834654 ]

Javier Puerto commented on COCOON-2340:
---------------------------------------

Hi,

I've more familiar with the 2.2 version. I've experienced issues like you described and it was by the following reasons, now works like a charm :). So, could you check the following?

* The components in the pipeline was declared as singleton instead of prototype. Some components are not thread safe, use prototype scope to avoid this problem.
* The version 2.2 is using an older version of Apache Xalan. Upgrade to 2.2.1 or upgrade Xalan to latest, see COCOON-2156.

Anyway, could you provide an example to reproduce the problem if it persists?

Salu2.

Post by Ivan Lagunov (JIRA)
XMLByteStreamCompiler in not thread-safe
----------------------------------------
Key: COCOON-2340
URL: https://issues.apache.org/jira/browse/COCOON-2340
Project: Cocoon
Issue Type: Bug
Components: * Cocoon Core
Affects Versions: 2.2
Reporter: Ivan Lagunov
Caused by: java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at org.apache.cocoon.components.sax.XMLByteStreamCompiler.getSAXFragment(XMLByteStreamCompiler.java:61)
at org.apache.cocoon.components.pipeline.impl.ExpiresCachingProcessingPipeline.processXMLPipeline(ExpiresCachingProcessingPipeline.java:152)
After debugging I've found the root cause. It happens due to multi-threading when one thread allocates newbuf array, then another thread increments bufCount in write method and then the first thread fails on System.arraycopy invocation.
I suggest making getSAXFragment and write methods synchronized. It should resolve the issue. If it helps, I'll provide a patch later.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

[Ivan Lagunov (JIRA)]

[ https://issues.apache.org/jira/browse/COCOON-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834764#comment-13834764 ]

Ivan Lagunov commented on COCOON-2340:
--------------------------------------

Hi Javier,

I would like to try Cocoon 2.2.1 first but could you say where it's located? I suppose it must be under trunk here http://svn.apache.org/repos/asf/cocoon/trunk/ ? Is it stable enough for Production usage? I mean it's written here that SNAPSHOTS are not tested and not guaranteed to even build cleanly: https://cocoon.apache.org/mirror.cgi

Ivan

Post by Ivan Lagunov (JIRA)
XMLByteStreamCompiler in not thread-safe
----------------------------------------
Key: COCOON-2340
URL: https://issues.apache.org/jira/browse/COCOON-2340
Project: Cocoon
Issue Type: Bug
Components: * Cocoon Core
Affects Versions: 2.2
Reporter: Ivan Lagunov
Caused by: java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at org.apache.cocoon.components.sax.XMLByteStreamCompiler.getSAXFragment(XMLByteStreamCompiler.java:61)
at org.apache.cocoon.components.pipeline.impl.ExpiresCachingProcessingPipeline.processXMLPipeline(ExpiresCachingProcessingPipeline.java:152)
After debugging I've found the root cause. It happens due to multi-threading when one thread allocates newbuf array, then another thread increments bufCount in write method and then the first thread fails on System.arraycopy invocation.
I suggest making getSAXFragment and write methods synchronized. It should resolve the issue. If it helps, I'll provide a patch later.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

[Ivan Lagunov (JIRA)]

[ https://issues.apache.org/jira/browse/COCOON-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834764#comment-13834764 ]

Ivan Lagunov edited comment on COCOON-2340 at 11/28/13 12:15 PM:
-----------------------------------------------------------------

Hi Javier,

I would like to try Cocoon 2.2.1 first but could you say where it's located? I suppose it must be under trunk here http://svn.apache.org/repos/asf/cocoon/trunk/ ? Is it stable enough for Production usage? It's written here that SNAPSHOTS are not tested and not guaranteed to even build cleanly: https://cocoon.apache.org/mirror.cgi

Regarding singleton components in the pipeline. I'm using standard Cocoon components, you can see the exception is coming from ExpiresCachingProcessingPipeline. Did you mean that some Cocoon 2.2 components are declared as singletons? Because I don't think I can fix it in my code.

Ivan


was (Author: lagivan):
Hi Javier,

I would like to try Cocoon 2.2.1 first but could you say where it's located? I suppose it must be under trunk here http://svn.apache.org/repos/asf/cocoon/trunk/ ? Is it stable enough for Production usage? I mean it's written here that SNAPSHOTS are not tested and not guaranteed to even build cleanly: https://cocoon.apache.org/mirror.cgi

Ivan

Post by Ivan Lagunov (JIRA)
XMLByteStreamCompiler in not thread-safe
----------------------------------------
Key: COCOON-2340
URL: https://issues.apache.org/jira/browse/COCOON-2340
Project: Cocoon
Issue Type: Bug
Components: * Cocoon Core
Affects Versions: 2.2
Reporter: Ivan Lagunov
Caused by: java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at org.apache.cocoon.components.sax.XMLByteStreamCompiler.getSAXFragment(XMLByteStreamCompiler.java:61)
at org.apache.cocoon.components.pipeline.impl.ExpiresCachingProcessingPipeline.processXMLPipeline(ExpiresCachingProcessingPipeline.java:152)
After debugging I've found the root cause. It happens due to multi-threading when one thread allocates newbuf array, then another thread increments bufCount in write method and then the first thread fails on System.arraycopy invocation.
I suggest making getSAXFragment and write methods synchronized. It should resolve the issue. If it helps, I'll provide a patch later.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

[Javier Puerto (JIRA)]

[ https://issues.apache.org/jira/browse/COCOON-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13835015#comment-13835015 ]

Javier Puerto commented on COCOON-2340:
---------------------------------------

Hi Ivan,

You are right, I thought that the 2.2.1 was released but it's not. You can apply the patch from COCOON-2156 to the tag http://svn.apache.org/repos/asf/cocoon/tags/cocoon-2.2/
About the singleton components, you can try yourself if you redeclare the Cocoon component with a different name and then use this one instead of the default one in the core.
Anyway you should provide an example to reproduce the issue so I can take a look but I will not be able to review it until the weekend.

Salu2.

Post by Ivan Lagunov (JIRA)
XMLByteStreamCompiler in not thread-safe
----------------------------------------
Key: COCOON-2340
URL: https://issues.apache.org/jira/browse/COCOON-2340
Project: Cocoon
Issue Type: Bug
Components: * Cocoon Core
Affects Versions: 2.2
Reporter: Ivan Lagunov
Caused by: java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at org.apache.cocoon.components.sax.XMLByteStreamCompiler.getSAXFragment(XMLByteStreamCompiler.java:61)
at org.apache.cocoon.components.pipeline.impl.ExpiresCachingProcessingPipeline.processXMLPipeline(ExpiresCachingProcessingPipeline.java:152)
After debugging I've found the root cause. It happens due to multi-threading when one thread allocates newbuf array, then another thread increments bufCount in write method and then the first thread fails on System.arraycopy invocation.
I suggest making getSAXFragment and write methods synchronized. It should resolve the issue. If it helps, I'll provide a patch later.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

[Javier Puerto (JIRA)]

[ https://issues.apache.org/jira/browse/COCOON-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Javier Puerto reassigned COCOON-2340:
-------------------------------------

Assignee: Javier Puerto

Post by Ivan Lagunov (JIRA)
XMLByteStreamCompiler in not thread-safe
----------------------------------------
Key: COCOON-2340
URL: https://issues.apache.org/jira/browse/COCOON-2340
Project: Cocoon
Issue Type: Bug
Components: * Cocoon Core
Affects Versions: 2.2
Reporter: Ivan Lagunov
Assignee: Javier Puerto
Caused by: java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at org.apache.cocoon.components.sax.XMLByteStreamCompiler.getSAXFragment(XMLByteStreamCompiler.java:61)
at org.apache.cocoon.components.pipeline.impl.ExpiresCachingProcessingPipeline.processXMLPipeline(ExpiresCachingProcessingPipeline.java:152)
After debugging I've found the root cause. It happens due to multi-threading when one thread allocates newbuf array, then another thread increments bufCount in write method and then the first thread fails on System.arraycopy invocation.
I suggest making getSAXFragment and write methods synchronized. It should resolve the issue. If it helps, I'll provide a patch later.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

[Javier Puerto (JIRA)]

[ https://issues.apache.org/jira/browse/COCOON-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13835758#comment-13835758 ]

Javier Puerto commented on COCOON-2340:
---------------------------------------

Hi Ivan,

By the error, you are using a pipeline with expires caching enabled and the problem seems to be retrieving an XML fragment (CInclude transformer?). With this information I can't investigate further, I've also used with others projects the expires caching in pipelines without any concurrency issues so I recommend you to provide a small pipeline fragment where the problem can be reproduced so we can take a look. Following the advices I've said in the previous comment, Apache Cocoon 2.2 should not have any concurrency problems.

To found the possible cause you could use the "ab" tool (ApacheBench [1]) that comes with the Apache HTTPD server. Then comment everything that involves with your pipeline leaving just the generator and the serializer and start uncommenting and testing with "ab" tool with high concurrency, this way you can isolate the problematic component or pipeline.

Good luck and salu2.

[1] http://httpd.apache.org/docs/2.2/programs/ab.html

Post by Ivan Lagunov (JIRA)
XMLByteStreamCompiler in not thread-safe
----------------------------------------
Key: COCOON-2340
URL: https://issues.apache.org/jira/browse/COCOON-2340
Project: Cocoon
Issue Type: Bug
Components: * Cocoon Core
Affects Versions: 2.2
Reporter: Ivan Lagunov
Caused by: java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at org.apache.cocoon.components.sax.XMLByteStreamCompiler.getSAXFragment(XMLByteStreamCompiler.java:61)
at org.apache.cocoon.components.pipeline.impl.ExpiresCachingProcessingPipeline.processXMLPipeline(ExpiresCachingProcessingPipeline.java:152)
After debugging I've found the root cause. It happens due to multi-threading when one thread allocates newbuf array, then another thread increments bufCount in write method and then the first thread fails on System.arraycopy invocation.
I suggest making getSAXFragment and write methods synchronized. It should resolve the issue. If it helps, I'll provide a patch later.

--
This message was sent by Atlassian JIRA
(v6.1#6144)