[ZODB-Dev] Use of fsync in FileStorage
Christian Robottom Reis
kiko at async.com.br
Thu Jul 29 15:51:08 EDT 2004
On Wed, Jul 28, 2004 at 11:55:13PM -0600, Shane Hathaway wrote:
> On Wednesday 28 July 2004 15:47 pm, Tim Peters wrote:
> > Any use of os.fsync is clearly a txn-rate disaster on my WinXP box. It
> > would be great if readers here tried it on their boxes and reported
> > results: various Linux flavors with various filesystems, Solaris variants,
> > Windows boxes with "serious" disk systems, whatever matters to you in
> > practice.
>
> Here are my results for 10,000 transactions on a 1 GHz Athlon, Linux desktop
> using Reiserfs on a 40 GB drive.
>
> One fsync: 76.3015 seconds, 131.059 txn/sec
> Two fsyncs: 86.915 seconds, 115.055 txn/sec
> No fsyncs: 9.38018 seconds, 1066.08 txn/sec
I made some extra timings with vmstat output for the run on the server I
reported on before (raid-1 on lvd 10K scsi drives/adaptec 29160):
No fsyncs: 8.29973 seconds, 1204.86 txn/sec
procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
1 0 0 177116 33720 74020 591004 0 0 0 0 163 602 8 3 89
3 0 0 177116 28608 74020 592648 0 0 168 0 166 3385 83 16 2
3 0 0 177116 26420 74052 594220 0 0 0 148 206 2601 73 27 0
3 0 7 177116 24228 74096 595876 0 0 0 2808 230 3354 78 22 0
3 0 0 177116 22160 74132 597768 0 0 0 205 250 2411 84 16 0
3 0 0 177116 20276 74132 599548 0 0 16 0 159 3301 84 16 0
3 0 0 177116 18344 74140 601424 0 0 0 0 217 3096 83 17 0
5 0 0 177116 16156 74164 603284 0 0 84 80 215 2581 82 18 0
4 0 2 177116 14504 74184 604776 0 0 0 5639 312 2571 66 17 17
3 0 0 177116 19348 74192 605256 0 0 0 108 174 1212 24 8 69
One fsync: 11.4103 seconds, 876.397 txn/sec
2 0 0 177116 13312 60244 621440 0 0 0 0 147 3325 25 9 66
3 0 0 177116 13400 59748 622140 0 0 0 0 145 8654 76 24 0
3 0 0 177116 13288 58960 623200 0 0 0 0 133 8601 66 34 0
3 0 0 177116 13568 58016 623968 0 0 0 1896 231 8347 76 24 0
3 0 0 177116 13440 57076 625092 0 0 0 0 128 8457 66 34 0
2 0 0 177116 14056 56268 625312 0 0 0 0 138 8417 72 28 0
3 0 0 177116 12748 56268 626584 0 0 0 0 175 8391 71 29 0
3 0 0 177116 12708 55672 627220 0 0 4 0 147 8376 82 18 0
3 0 4 177116 12580 55308 627616 0 0 0 4552 474 7974 64 36 0
3 0 0 177116 12532 55288 627668 0 0 0 696 383 8178 72 28 0
3 0 0 177116 13264 55288 627044 0 0 0 0 133 8265 76 24 0
2 0 0 177116 17240 55296 627304 0 0 0 1312 172 7996 80 19 1
1 0 0 177116 17236 55296 627304 0 0 0 0 147 591 0 2 98
Two fsyncs: 12.8292 seconds, 779.474 txn/sec
1 0 0 178652 99724 52976 549376 0 0 0 0 128 563 0 3 97
2 0 0 178652 95168 53024 549724 0 0 0 279 205 8833 40 23 37
2 0 0 178652 94072 53024 550820 0 0 0 0 118 14246 75 25 0
2 0 0 178652 92960 53024 551932 0 0 0 0 119 14181 69 31 0
3 0 0 178652 91828 53028 553060 0 0 4 0 117 14270 74 26 0
3 0 0 178648 90708 53028 554184 0 0 0 0 156 14003 65 35 0
2 0 0 178648 89568 53052 555284 0 0 0 2688 196 13579 70 30 0
3 0 0 178648 88416 53052 556412 0 0 0 0 119 13916 70 30 0
3 0 0 178648 87260 53052 557548 0 0 0 0 139 13815 69 31 0
4 0 0 178648 86092 53060 558700 0 0 0 0 125 13921 63 38 0
4 0 0 178648 84924 53060 559848 0 0 4 0 138 13802 68 32 0
3 0 0 178648 82784 53092 560884 0 0 0 3087 352 12297 63 38 0
4 0 0 178648 81624 53092 562028 0 0 0 0 138 13464 71 29 0
4 0 0 178648 80444 53092 563196 0 0 0 0 125 13570 74 26 0
1 0 0 178648 84620 53092 563316 0 0 0 0 121 1899 6 9 85
Doesn't really seem to make *that* much of a difference; this may have
to do with VFS fsync behaviour when using raid-1, or perhaps the write
behaviour on these specific hard drives -- I am surprised so many others
saw so much fluctuation in times. I might run this on a similar server
that I have remote access to if Tim says he wants me to.
Take care,
--
Christian Robottom Reis | http://async.com.br/~kiko/ | [+55 16] 3361 2331
More information about the ZODB-Dev
mailing list