From: "Steven Levine" <ecs-isp@2rosenthals.com>
Message-ID: <67264757.3.mr2ice.fgrirsq@earthlink.net>
Date: Sat, 02 Nov 2024 08:37:59 -0700
To: "eCS ISP Mailing List" <ecs-isp@2rosenthals.com>
In-Reply-To: <list-11202333@2rosenthals.com>
Subject: Re: [eCS-ISP] clamscan issue - directories with a lot of files
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-15
Content-Transfer-Encoding: quoted-printable

In <list-11202333@2rosenthals.com>, on 11/01/24
   at 12:05 PM, "Massimo S." <ecs-isp@2rosenthals.com> said:

HI Massimo,

>>> damn, i just deleted a debug.txt output for mistake.. :-(

Oops.

>>> anyway it was a 4GB file, i don't have it in the bkups,
>>> maybe ZIP is not able to add a so large file

Depends on the version of zip.

>>> and i guess that the issue of "Can't read file ERROR"
>>> is not related to a directory with a lot of files inside
>>> since it appears allmost the times in "big dirs", but
>>> it may happens that clamscan keep on working again in the same dir
>>> e.g. in this case:

>>> X:/weasel/MailRoot/mydomain.com/myemail/OLAYI6.MSG: Can't read file E=
RROR
>>> X:/weasel/MailRoot/mydomain.com/myemail/OLAZFS.MSG: Can't read file E=
RROR
>>> X:/weasel/MailRoot/mydomain.com/myemail/OLB25S.MSG: Can't read file E=
RROR
>>> X:/weasel/MailRoot/mydomain.com/myemail/OLB3TA.MSG: Can't read file E=
RROR
>>> X:/weasel/MailRoot/mydomain.com/myemail/OLB5K9.MSG: Can't read file E=
RROR

>> i'm able to reproduce the debug output
>> uncompressed it's 5,6GB..=A0 doh=A0 :-)
>> compressed is 2,6GB

For a file this large, I recommed you use 7z.

>> i can put it on an ftp
>> let me know

Not yet.  I busy with some other projects.

>when it show the errors i see a lot of these in the output on the screen=
:
>LibClamAV Error: fmap_readpage: pread error: Not enough memory

That message pretty much explains itself.

>the VM has 4GB ram assigned (i know /2 see only about 3,3GB) and
>vaddresslimit is 3072

That's as good as it's going to get.

Clamscan is a memory hog.  There are two issues - overall memory usage an=
d
fragmentation.

I don't seem to have a copy of your clamscan command line.

I suspect you are going have to break the clamscan run into smaller chunk=
s
to avoid the memory issues.

Steven

-- 
----------------------------------------------------------------------
"Steven Levine" <steve53@earthlink.net>  Warp/DIY/BlueLion etc.
www.scoug.com www.arcanoae.com www.warpcave.com
----------------------------------------------------------------------