Welcome to Squishdot Websites Squishdot How-To
 about
 search
 post article
 Documentation
 Mailing Lists
 Bug Tracking
 Development
 Installation
 Upgrading
 Download
 admin
 rdf

 main
How to deal with many postings
Squishdot Posted by on Thursday October 14, 05:17AM, 1999
from the squish-in-chinese dept. dept.
I have set up a Squishdot site (excellent software Butch) in chinese. We have modified the DTML as well as Squishdot.py to add some extra functionality. However as the site is becoming more popular we have encountered the following problems: 1. The python process is consuming too much RAM >256Mb of RAM whereas the whole Data.fs is only 7Mb. I am using Zope-2b1 so the memory leak could be with Zope or with Squishdot, however we are running one more Zope-2b1 site (non-squishdot) and we never got such problems.

2. In two months we have accumulated 900 postings (are we the largest Squishdot site?) and it gets very difficult to maintain. Moreover a search can return too many results and we would like to use some batch viewing. The way the search function is currently implemented does not allow the use of the DTML batch processing constructs. (This is not a Squishdot problem per se, and I am already working to solve it, but if someone has an easy solution please let me know). I would have *really* liked it if the postings were normal DTML Documents with associated properties, seperated in folders according to subject or even better by month. That would have solved our third problem (also mentioned elsewhere on the Squishdot site) of migrating our postings to the newer version.

In summary: I am going to make heavy modifications to Squishdot so I might as well ask all of you for advice. For ease of management of large Squishdot sites, do you think is a good idea to move the postings into folders by month? Butch, would this break too much of the internal structure? If not do you think is a good idea to use DTML Douments to represent postings so we can use Zope's regular import/export mechanisms? If not then I am willing to write an export/import function in XML.

<  |  >

 
Related Links

The Fine Print: The following comments are owned by whoever posted them.
( Reply )
Re: How to deal with many postings
by on Saturday October 16, 09:08PM, 1999
I have no clue for the memory leak; Technocrat.net, has a bigger number of postings (1400+) but Bruce has'nt had (or at least, has'nt told me) memory usage problems -- of course he might have slightly different versions and configuration of the software.

One of the things I've learned since I started is that I need to improve the moderation interface -- the tree-like model is not sufficient for large volumes of postings and I plan to modify this substantially in the future.

If you're planning to write an export/import mechanism, I'd love to work with you on this. I've been trying to do this myself and have been exploring various things(XML and stuff) -- but I haven't gotten very far.

If you are going to work on this, please email me directly or use the Squishdot mailing list so I can share some of my ideas as well and hopefully put this into the mainstream Squishdot code ASAP

BTW, can we access your chinese squishdot site?

--Butch
[ Reply to this ]
  • Re: How to deal with many postings
    by on Tuesday October 19, 09:34PM, 1999
    Zope or Squishdot might have a memory leak. A few times, Technocrat has become sluggish and a restart of Zope has fixed it.


    Bruce
    [ Reply to this ]
    • Re: How to deal with many postings
      by on Wednesday October 20, 02:40PM, 1999
      This is how we are dealing with it right now.

      Restart Zope every couple of days.

      The site is currently at:

      www.yifannet.com:8080/News

      and it serves about 30000 requests per day.
      [ Reply to this ]

 
The Fine Print: The following comments are owned by whoever posted them.
( Reply )
Powered by Zope  Squishdot Powered 		  "Any system that depends on reliability is unreliable." -- Nogg's Postulate
All trademarks and copyrights on this page are owned by their respective companies. Comments are owned by the Poster. The Rest ©1999 , ©2000-2002 .