Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More efficient JSON codecs for sets and maps #778

Merged

Conversation

plokhotnyuk
Copy link
Contributor

@plokhotnyuk plokhotnyuk commented Jan 10, 2025

The smaller set or map the better speed up of JSON parsing and serialization.

Also, this commit improves the internal implementation of instantiations of encoders and decoders to reduce number of instances (by removing of redundant parameter for discriminator) and speed up their instantiation for complex schemas.

Below are throughput results before and after proposed changes for different lengths of sets and maps with Scala 3 and JDK 21 on Intel® Core™ i7-11800H CPU @ 2.3GHz (max 4.6GHz).

the main branch (before)

Benchmark                                  (size)   Mode  Cnt         Score         Error  Units
MapOfIntsToBooleansReading.zioJson              1  thrpt    5  21756869.827 ±  552828.024  ops/s
MapOfIntsToBooleansReading.zioJson             10  thrpt    5   1512830.064 ±   95335.512  ops/s
MapOfIntsToBooleansReading.zioJson            100  thrpt    5    165645.824 ±    3618.193  ops/s
MapOfIntsToBooleansReading.zioJson           1000  thrpt    5     16674.642 ±    2271.084  ops/s
MapOfIntsToBooleansReading.zioSchemaJson        1  thrpt    5  11672087.963 ±  387349.141  ops/s
MapOfIntsToBooleansReading.zioSchemaJson       10  thrpt    5   1214992.904 ±   26255.066  ops/s
MapOfIntsToBooleansReading.zioSchemaJson      100  thrpt    5    117287.352 ±    2603.652  ops/s
MapOfIntsToBooleansReading.zioSchemaJson     1000  thrpt    5     15664.449 ±     668.689  ops/s
MapOfIntsToBooleansWriting.zioJson              1  thrpt    5  18492927.580 ±  313153.941  ops/s
MapOfIntsToBooleansWriting.zioJson             10  thrpt    5   2505178.392 ±   64899.744  ops/s
MapOfIntsToBooleansWriting.zioJson            100  thrpt    5    253449.887 ±   15691.321  ops/s
MapOfIntsToBooleansWriting.zioJson           1000  thrpt    5     21806.744 ±     174.455  ops/s
MapOfIntsToBooleansWriting.zioSchemaJson        1  thrpt    5  12203450.362 ±  254378.831  ops/s
MapOfIntsToBooleansWriting.zioSchemaJson       10  thrpt    5   1076284.865 ±   17141.827  ops/s
MapOfIntsToBooleansWriting.zioSchemaJson      100  thrpt    5    126766.898 ±    2707.606  ops/s
MapOfIntsToBooleansWriting.zioSchemaJson     1000  thrpt    5     12703.254 ±     207.542  ops/s
SetOfIntsReading.zioJson                        1  thrpt    5  33212379.612 ± 1103032.456  ops/s
SetOfIntsReading.zioJson                       10  thrpt    5   2551729.835 ±   61829.704  ops/s
SetOfIntsReading.zioJson                      100  thrpt    5    286412.914 ±    5489.927  ops/s
SetOfIntsReading.zioJson                     1000  thrpt    5     31191.133 ±     266.829  ops/s
SetOfIntsReading.zioSchemaJson                  1  thrpt    5  16078985.333 ±  348626.181  ops/s
SetOfIntsReading.zioSchemaJson                 10  thrpt    5   2044182.440 ±   25687.309  ops/s
SetOfIntsReading.zioSchemaJson                100  thrpt    5    259775.283 ±     877.448  ops/s
SetOfIntsReading.zioSchemaJson               1000  thrpt    5     25538.931 ±    3987.873  ops/s
SetOfIntsWriting.zioJson                        1  thrpt    5  29801943.298 ± 1048553.921  ops/s
SetOfIntsWriting.zioJson                       10  thrpt    5   5724048.138 ±  149662.453  ops/s
SetOfIntsWriting.zioJson                      100  thrpt    5    625381.088 ±    9706.680  ops/s
SetOfIntsWriting.zioJson                     1000  thrpt    5     67492.040 ±    2274.049  ops/s
SetOfIntsWriting.zioSchemaJson                  1  thrpt    5  17078533.044 ±  405958.003  ops/s
SetOfIntsWriting.zioSchemaJson                 10  thrpt    5   4190168.182 ±  106019.142  ops/s
SetOfIntsWriting.zioSchemaJson                100  thrpt    5    507224.173 ±   14499.742  ops/s
SetOfIntsWriting.zioSchemaJson               1000  thrpt    5     49905.827 ±    1555.988  ops/s

this PR (after)

Benchmark                                  (size)   Mode  Cnt         Score        Error  Units
MapOfIntsToBooleansReading.zioJson              1  thrpt    5  21596104.721 ± 300558.574  ops/s
MapOfIntsToBooleansReading.zioJson             10  thrpt    5   1384031.097 ±  37621.088  ops/s
MapOfIntsToBooleansReading.zioJson            100  thrpt    5    161117.800 ±   9590.403  ops/s
MapOfIntsToBooleansReading.zioJson           1000  thrpt    5     15502.791 ±   1375.775  ops/s
MapOfIntsToBooleansReading.zioSchemaJson        1  thrpt    5  20743870.357 ± 441574.069  ops/s
MapOfIntsToBooleansReading.zioSchemaJson       10  thrpt    5   1407598.714 ±  38423.696  ops/s
MapOfIntsToBooleansReading.zioSchemaJson      100  thrpt    5    157121.477 ±   4877.024  ops/s
MapOfIntsToBooleansReading.zioSchemaJson     1000  thrpt    5     15377.942 ±    242.475  ops/s
MapOfIntsToBooleansWriting.zioJson              1  thrpt    5  19024503.909 ± 255670.834  ops/s
MapOfIntsToBooleansWriting.zioJson             10  thrpt    5   2345766.229 ±  24802.164  ops/s
MapOfIntsToBooleansWriting.zioJson            100  thrpt    5    264292.723 ±   3903.843  ops/s
MapOfIntsToBooleansWriting.zioJson           1000  thrpt    5     25895.774 ±   1123.635  ops/s
MapOfIntsToBooleansWriting.zioSchemaJson        1  thrpt    5  18765038.429 ± 464321.261  ops/s
MapOfIntsToBooleansWriting.zioSchemaJson       10  thrpt    5   2393788.294 ±  59654.336  ops/s
MapOfIntsToBooleansWriting.zioSchemaJson      100  thrpt    5    288604.915 ±  11645.194  ops/s
MapOfIntsToBooleansWriting.zioSchemaJson     1000  thrpt    5     26346.191 ±    490.979  ops/s
SetOfIntsReading.zioJson                        1  thrpt    5  33046626.630 ± 678242.427  ops/s
SetOfIntsReading.zioJson                       10  thrpt    5   2531610.299 ±  92937.683  ops/s
SetOfIntsReading.zioJson                      100  thrpt    5    284806.595 ±   4003.228  ops/s
SetOfIntsReading.zioJson                     1000  thrpt    5     31622.096 ±    649.715  ops/s
SetOfIntsReading.zioSchemaJson                  1  thrpt    5  34429744.324 ± 343072.401  ops/s
SetOfIntsReading.zioSchemaJson                 10  thrpt    5   2604395.212 ±  45600.816  ops/s
SetOfIntsReading.zioSchemaJson                100  thrpt    5    296153.841 ±   5787.944  ops/s
SetOfIntsReading.zioSchemaJson               1000  thrpt    5     30403.317 ±   1354.201  ops/s
SetOfIntsWriting.zioJson                        1  thrpt    5  29892042.438 ± 493124.725  ops/s
SetOfIntsWriting.zioJson                       10  thrpt    5   5771840.714 ± 112021.148  ops/s
SetOfIntsWriting.zioJson                      100  thrpt    5    603232.965 ±   7424.742  ops/s
SetOfIntsWriting.zioJson                     1000  thrpt    5     67880.881 ±   1290.703  ops/s
SetOfIntsWriting.zioSchemaJson                  1  thrpt    5  30902664.436 ± 884786.856  ops/s
SetOfIntsWriting.zioSchemaJson                 10  thrpt    5   6023396.560 ±  83308.203  ops/s
SetOfIntsWriting.zioSchemaJson                100  thrpt    5    609386.479 ±   5308.527  ops/s
SetOfIntsWriting.zioSchemaJson               1000  thrpt    5     66243.724 ±   1984.984  ops/s

@plokhotnyuk plokhotnyuk force-pushed the more-efficient-json-codecs-for-sets-and-maps branch from e980989 to e509187 Compare January 10, 2025 19:50
@plokhotnyuk plokhotnyuk marked this pull request as ready for review January 10, 2025 20:49
@plokhotnyuk plokhotnyuk requested a review from a team as a code owner January 10, 2025 20:49
@jdegoes jdegoes merged commit 64fc155 into zio:main Jan 11, 2025
26 checks passed
@plokhotnyuk plokhotnyuk deleted the more-efficient-json-codecs-for-sets-and-maps branch January 11, 2025 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants