This message was deleted.

Can you make a comparison with writing directly to the disk?

yes

# fio --name=write_bandwidth_test  --filename=/dev/nvme0n1 --filesize=12500G --time_based --ramp_time=2s --runtime=1m --ioengine=libaio --direct=1 --verify=0 --randrepeat=0 --bs=1M --iodepth=64 --rw=write --numjobs=16 --cpus_allowed 1 --cpus_allowed_policy=split --offset_increment=100G --group_reporting=1
write_bandwidth_test: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.28
Starting 16 processes
Jobs: 16 (f=16): [W(16)][0.0%][w=1365MiB/s][w=1365 IOPS][eta 01d:19h:04m:04s]
write_bandwidth_test: (groupid=0, jobs=16): err= 0: pid=2397208: Wed Jun 26 12:51:23 2024
  write: IOPS=1363, BW=1381MiB/s (1448MB/s)(81.1GiB/60129msec); 0 zone resets
    slat (usec): min=50, max=136556, avg=11704.30, stdev=6222.22
    clat (msec): min=101, max=913, avg=734.25, stdev=47.10
     lat (msec): min=106, max=914, avg=745.95, stdev=47.32
    clat percentiles (msec):
     |  1.00th=[  625],  5.00th=[  709], 10.00th=[  709], 20.00th=[  718],
     | 30.00th=[  726], 40.00th=[  735], 50.00th=[  735], 60.00th=[  743],
     | 70.00th=[  751], 80.00th=[  751], 90.00th=[  768], 95.00th=[  768],
     | 99.00th=[  785], 99.50th=[  793], 99.90th=[  810], 99.95th=[  818],
     | 99.99th=[  844]
   bw (  MiB/s): min= 1232, max= 1564, per=99.00%, avg=1367.04, stdev= 3.22, samples=1920
   iops        : min= 1232, max= 1564, avg=1366.96, stdev= 3.22, samples=1920
  lat (msec)   : 250=0.37%, 500=0.41%, 750=73.25%, 1000=27.19%
  cpu          : usr=0.44%, sys=0.42%, ctx=291248, majf=0, minf=932
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=0,82002,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
  WRITE: bw=1381MiB/s (1448MB/s), 1381MiB/s-1381MiB/s (1448MB/s-1448MB/s), io=81.1GiB (87.1GB), run=60129-60129msec

Disk stats (read/write):
  nvme0n1: ios=100/681843, merge=0/0, ticks=54/63331635, in_queue=63331689, util=99.06%

Could you use the same tool so that we can compare the output?

yes sure i will do it later today, thx