DMFA forum redirected.

[llearch n'n'daCorna]

Skwerty, I'm happy to host the full version, if you want.

I'm sure there's a bunch of others who'd also be willing...

[skwerly]

Aridas: m'kay. Eeee.

ttearch ,  do you have a size limit?  The images are going to take up at least a couple gig.

Speaking of images... I don't know why I didn't consider this before, but there's a huge mess of copyrighted works here, especially in the fan art threads.  Since they were posted in the forum, does that make them public domain and freely distributable?  I doubt I'll be able to find three years worth of artists for their blessings.

[llearch n'n'daCorna]

As far as size limits go, approx 30Gb, I think.

I suspect, as far as the forum goes, posting in the forum makes the content owned by the hoster of the forum, and arguing that you really want it taken down is probably not going to fly.

Technically, one should get permission for re-hosting it, but since there's no -explicit- "don't rehost this" disclaimers on any of the forum posts, the -implicit- rules are basically those of the wide web, ie, anything goes.

IANAL, but I'd expect that, as long as we weren't making money off it, and were willing to take down any posts that anyone wanted removed (which, for my part, I am) we'd probably get away with it. You may wish to add a front page that documents that...

[Damaris]

Posting it for public consumption does not make it public domain.  The original owners still own the copyright, and would be well within their rights to demand anything being taken down (especially since they won't be able to edit it out of their posts anymore.)

However, since you guys are hosting a mirror of stuff that was originally posted somewhere else, and are not altering the forum in any significant way, it should be fine.  Like you said, you should be able to get away with it, especially if credit is given on everything (which it would, since it's a mirror)

[llearch n'n'daCorna]

Oh, -yes-

If any of the original posters was to appear and ask for stuff to be removed, removed it would be. No arguments, no questions - other than confirming the veractiy of the person claiming to own the content, and that's merely a formality, since I prefer to err on the side of safety - in this, at least, if nothing else...

However, my point was, the copyright may, depending on the EULA that was clicked through to register with Nice, belong to Nice, rather than to the original poster.

The author has rights, but so does the Nice forums, and I don't recall anything twisty about the EULA, but then, I figure anything I post on the web is fair game to -everyone- who can see it, and there's technically nothing I can do to stop anyone taking copies, except asking politely. *shrug* So attempting to stop people from taking copies is just going to stop people from being able to see the thing I've put up, and since I put stuff up so people can see it....


*cough* if that's not enough to explain, nothing is. :-)

[skwerly]

Eep!  I don't want to get into a discussion on the details of copyright law or anything.. I was mainly concerned for the artists.  We'd be redistributing art that others drew, and I don't want to upset those others.

As for The Nice, their registration page says they don't review the posts, and is not responsible for the content.  "You remain solely responsible for the content of your messages."  So, copyright law aside, I'm taking that to mean I should worry about the artists, not The Nice or Purrsia.

I'll go with the "easier to ask forgiveness than permission" route, I suppose.  If a poster wants his/her post or image removed, it gets removed, no questions asked.  They'd have to actually notice the mirror first, but I'll assume there's enough artists/posters reading this that word will get around quickly enough.

If anyone else has concerns about this, please speak up!  (If I'm the only one, then I'll just write it off as paranoia and move on to my next crisis.  I have a list.)

[Aridas]

Well, since we're not putting it up anywhere other than where it was already put up, that point should be done with. The fact that we're only preserving what's about to disappear should mean that there's nothing going on.

[llearch n'n'daCorna]

So.. what's next on the list, skwerly? :-)

[skwerly]

llearch sez:

So.. what's next on the list, skwerly? :-)

Getting the silly thing to actually work!  That's enough of a crisis for now.  1900 pages archived, about 6000 to go.  3 more days of downloading, then another day or two to mash it flat and fix links.  The semicolons in the filenames give Firefox fits.

Wheeee! *thud*

[skwerly]

(Hmm.. Do double posts count as A Bad Thing if they're 4 days apart?)

Update:

Archive complete!  Burning a backup now in case I screw up the repairs.  Gimme a few days to make it mirrorable.

[Tapewolf]

Archive complete!  Burning a backup now in case I screw up the repairs.  Gimme a few days to make it mirrorable.

Cool.  How big is it in total?

**EDIT**
And did you include other resources, such as music?  There are some files that I'm only keeping up because The Nice links to them.. when it goes away, I'll take them down since most of them were WIP files that have since been superceded.

[llearch n'n'daCorna]

If we're going to mirror it, supplying a compressed version would be significantly better in terms of bandwidth etc.

Not least - opening and closing more than 4000 files generates some overhead...

[skwerly]

@Tapewolf: It's currently weighing in at 5.3GB, but I suspect at least half of that is from duplicate images.  Since I had to archive each page separately, that means 8000 copies of navigation buttons, for example.  I'll have that cleared up soon enough.

Also, I archived only the text and embedded objects, and I chose not to have links point to the actual URL.  They all point to "where the local file would be if I grabbed it", and I didn't grab anything from links.  Because of all the navigation links, Teleport Pro kept choking on anything past the actual page I wanted.  It's why I had to create 8000 projects in the first place.

Now, if anyone wants to browse their favorite topics, download some linked images/mp3s/etc, and send them to me, I can probably add them to the archive with no problem.  All I'd need is the original filename.  It won't matter which topic or page the link came from.  Remember that I already have the embedded stuff.  If you can see it on the page, I have it.  If it's just a link, I don't have it.

@llearch: I know nothing about hosting, so I'll hand over the raw data and let you take care of compression and whatnot.  Is there anything HTMLish I should take into account while I change filenames and links around?  I'm removing the semicolons, as I said earlier.  I'm also removing cookie references and page headers/footers.  There's really no need for login/register/reply/etc links in an archive.

One thing that may or may not be a concern is all those links generating 404 errors, since they'll be looking at your server for the files.  I hadn't planned on doing anything about 'em, though.  Should I?

[llearch n'n'daCorna]

uh.

I meant for transporting it from where you are to where my server is. For that step, zipping the whole lot up into a single file, or even a bunch of files of reasonable size, and letting us download those large chunks, is far more efficient, networkwise, than downloading 8000 individual files.

As for the links... hmm. Uh. I guess we can ignore them, for the moment. Not a huge issue - I figure I can set up another domain and a vhost for it.

[Aridas]

I take it you didn't consider setting exceptions for Teleport?

[skwerly]

@llearch: D'oh! Well, that pretty much proves I know nothing of hosting, or I woulda understood that the first time.  I was going to send you a DVD, but I can upload a zip file, sure.

@Aridas: I tried just about every combination of settings I could think of, including using exceptions.  Some combos worked better than others, but none worked well enough to grab only what I wanted without hitting the 65k url limit.  That was about two days of experimentation.  Once I finally hit on the one-project-per-page idea, I was tired of testing, and I went with a somewhat minimal approach.

@Tapewolf: New estimate. After finding over 200,000 duplicate files (still deleting, ACDSee's having a blast), the archive will be down to about 1.5GB.