On Thu, Sep 13, 2007 at 08:10:35PM -0300, Alejandro Lozanoff wrote:
quoted text
> Thanks for your explanation and quick response, however with
> -Uusemymalloc it segfaults almost when it starts. At least it showed
> that the problem comes from that way, probably the mymalloc is worse
> than  the OpenBSD one. :P

memory use-wise it is not.
perhaps you have hit a geniune perl bug then.

quoted text
> We found what appears to be a workaround on awstats.
> Changing $tokenquery=||''; to $tokenquery='?'; after we traced what
> was awstats running when segfaulting.
> It's on awstats.pl line 6574
> ----- awstats 6.7 (build 1.892) (c) 2000-2007 Laurent Destailleur -----
> 
> It runs perfect and we didnt find any problem, but we don't have a clue
> as to why that is... We are still looking for a real solution
> 
> Thanks again,
> Alejandro.
> 
> mickey wrote:
> > On Thu, Sep 13, 2007 at 05:25:32PM -0300, Alejandro Lozanoff wrote:
> >   
> >> Hello list,
> >>     
> >
> > re
> >
> >   
> >> We recently updated a 3.7 machine running awstat(perl) to parse all our
> >> websites logs with the biggest being around 1GB.
> >> When parsing the big log it randomly segfaults on 4.1, 3.9 and 3.8, we
> >> tried new clean release installs and it still segfaults. On 3.7 it works
> >> flawlessly, on 3.8 which has the same perl version as 3.7 (5.8.6) it
> >> still segfaults. The problem is completely random but it tends to happen
> >> after its been running for a while as it doesnt happen on small logs (or
> >> the probability for it to happen on those files is too low )
> >>
> >> As the trace below (from the perl.core) shows, it's an out of bounds
> >> problem, then we remember about a change on 3.8:
> >>
> >> "malloc(3) has been rewritten to use the mmap(2) system call,
> >> introducing unpredictable allocation addresses and guard pages, which
> >> helps in detecting heap based buffer overflows and prevents various
> >> types of attacks."
> >>     
> >
> > yes. this increases memory fragmentation immensly resulting
> > in (practically) less virtual space available for data.
> > as an increased penalty (200-300%) for cpu consuption
> > on processes w/ lots (20M and more) malloc(3)ed memory...
> > as well increased demand for the physical memory that
> > on the overcommiting nature of it you perhaps observe.
> > a way around it is only to use perl malloc (sbrk-based)
> >
> > cu
> 

-- 
    paranoic mickey       (my employers have changed but, the name has remained)