FORUM MIGRATION SCHEDULED JAN 24th

as for moving the data, it is actually pretty simple, thing is ... if you are willing to rearrange 350664 Posts within 49236 Topics from 37469 registered members into the categories of the new forum, I will be more than happy to pass the task over to you guys.

I will do it, that is one of the things a computer can handle, now which forum system are you moving to so I can start to prepare, and when can I expect a copy of the data

not joking (I may not be successful but I will at least try my best, and I have moved a forum nearly as large over 4 systems to get from A to B with a rotten database)

I may not be successful but I will at least try my best, and I have moved a forum nearly as large over 4 systems to get from A to B with a rotten database

Be careful what you ask for, Osgeld... :wink:

Based on the blog posting (and I may be completely wrong about this), there seemed to be the implication that this current forum -isn't- using a DB back-end under Perl (flat text files, anyone?)...

If true, such a system could be a real nightmare to migrate without some serious script-fu to stuff the data from those files into a DB properly. It wouldn't be impossible, but it wouldn't be quick nor easy, either.

Good luck if you manage to pull the task!

:slight_smile:

it is a flat file system, but step 1 is to get it into SMF,which there are scripts available already, from there it can be converted to just about anything
:wink:

as for moving the data, it is actually pretty simple, thing is ... if you are willing to rearrange 350664 Posts within 49236 Topics from 37469 registered members into the categories of the new forum, I will be more than happy to pass the task over to you guys.

Yeah I'm willing to help with this.

So - how is "number of posts" going to be handled? Will everybody go back to "newbie" status? How will real newbies (ie - those who just bought their first Arduino Uno) know the "old timer god members" (those who have had an Arduino for a while, or those who have an original hand-soldered homebuilt ATMega8 board from way-back-when)?

Also a concern.

Presumably post counts could be manually edited in the system. Give everyone the post count they currently have.
Also what about 'member since' - is that going to be reverted? - Please no...

And my private messages?

Good Evening mowcius, you have 274 messages, and no new messages.

What's to happen with them if the system gets changed - can I still access them at least?

Mowcius

I would not really worry about putting things into new catagories, convert, stuff in a read only archive subsection and call it a day, but with conversion you have everything searchable in 1 system and usernames / passwords / private messages are preserved

1.3 cents

I would not really worry about putting things into new catagories, convert, stuff in a read only archive subsection and call it a day

Perhaps it would be possible for a user to transfer a thread when required from the archive - then they choose which subsection to put it under and it then starts a new thread (with the old posts included).

Not sure if that would be possible/useful.

Anyway - conversion is required...

Mowcius

depends on the system, most plain forum system it would involve a moderator action, maybe a request form, on more real things its quite easy but they are much more than just a forum

I think most of the comments you are making are pretty meaningful and I must admit some of them we had probably not given enough importance to.

Let me elaborate this, I will do it as a list, since it will allow for further commenting by those of you interested. I think the discussion at this level is very healthy, and even if I might not be able of sleeping in the incoming weeks, I will do my best to make your concerns come to a good end.

  1. the icons: dude, if you like them, I will make sure they are there, but please first give a try to the new ones :slight_smile:

  2. the current status and amount of messages posted: this is doable, we didn't have this into our list, but I can understand the importance of the status for you guys and I will talk to the team in order to try to make this happen, if not the day planned for the migration, then later. I must say our idea was to promote those of you really active to moderators, but that was of course a given

  3. transferring the previous posts to the new system (this is a longer issue, let me explain it)

Yabb is a file based system, it counts with 4 files per user: userinfo, user logs, user posts, and user messages. 37.000 users means 4 times more files ... IN A SINGLE FOLDER

The converters existing for moving from Yabb to SMF are NOT working, sorry for the SMF developers, but it took me a week to hack it and make it work again.

Right now we have stream-lined the process of migrating the users but not so much the posts. Mostly because we consider the new forum has to be structured in a completely different way. After 5 years with the current set of categories, we have gained quite some experience in the topics that matter the most to people and have identified the categories that seem to make the most sense.

The time it takes to migrate the 37.000 users to SMF is irrelevant, in my 2.3GHZ laptop I need about 3 hours. Then the database has to be filtered, because as explained, this is not just about the forum, this is about having a so-called Single Sign On system that will work all over the Arduino system: A single usernam and password to work on the forum, the wiki, the blog (the day we open it for external editors), other wikis, the store, and future services we have in our roadmap.

This means, our SMF is not really SMF, it is SMF made into our own identification system.

However, even if I think it is a perfectly valid point the one of having a single location where to search for information from the forum, we didn't conceive it like that in the first place. One possible solution, that cannot be scheduled for next Monday, but could be in a week more, is to make a category called "old" where to push in all the old messages into a single table, having the current categories as subcategories of that old forum. We could make it so that that table would not be editable but by moderators. Then users could request to move some of those topics to the new boards.

This is a lot of work, but if there are 15 volunteers, we could consider it. To me it will me I will sleep one week less this year, but it will be worth the effort.

Summarizing, I think the following things have to be considered:

  • moving the current user status to the new system
  • adding the current messages as a subcategory just to have them searchable
  • checking the emoticons and maybe use the current ones (just to please you above all the other 36.999 users :slight_smile: )

Something to think about:

  • letting a bunch of moderators access to a read-only archive to -upon request- move topics of importance to the new boards

Any other thoughts?

  • moving the current user status to the new system

Sounds good.

  • adding the current messages as a subcategory just to have them searchable

That sounds like a much better way to have it.

  • checking the emoticons and maybe use the current ones (just to please you above all the other 36.999 users Smiley )

Hehe - I see what he means - can't remember what the SMF ones are like atm but perhaps that is a discussion to have on a thread when the new forum turns up.

We could make it so that that table would not be editable but by moderators. Then users could request to move some of those topics to the new boards.

  • letting a bunch of moderators access to a read-only archive to -upon request- move topics of importance to the new boards

Yes - something like that sounds like it could be an acceptable way to go - not sure how 'requests' would be best submitted. Perhaps we can have a 'request' button on the 'archive' forum section.

Any other thoughts?

Private messages? Can they be stored at least or some way of backing them up - I won't have the time to try and back them up individually before a forum switch (and that would be one hell of a copy/paste job).

Maximum dimensions on images (preferably with auto-resize) so we don't get people posting a few thousand px images slowing it all down and making the threads look a mess. Thumbnails for images over ~600px wide is possibly the best way to go here.

This is a lot of work, but if there are 15 volunteers, we could consider it.

Count me in for helping.

Mowcius

not to be blunt but if your thinking of a single sign on system why not use a single system (ie joomla) to handle it all instead of patchworking all these little systems together through bridges and hacks which makes even a simple future upgrade (ie security) a nightmare as you fight with rehacking stuff and opps bridge 1.2.1 doesnt work with front end 1.1.2 and etc

count me in on helping too (if that were not evident)

(and for the record I have had the smf converter working for yabb 2.2.3 for a few months now, whenever it was you put up that poll / test forum :wink: though its not perfect, its never perfect)

What about a preview of what the new forum is going to look like?

It is pretty clear everyone has hopes and wishes for whatever is coming next. Not everyone is going to be happy. But it would take a lot of guesswork (and apparently worrying) if the interested parties could see what the new forum looks like.

May I suggest more local boards, like USA/MN so we can organize local gatherings to share experience face to face. Helps to know who does what in your area, don't you think so?

I've seen "migration" attempts where one forum was left frozen as an archive. It's horrible.

First of all, the huge number of posts in an existing forum should be taken as MORE of an incentive to move that data into the new forum. The more posts you have, the greater the value of that data set. If you had only a few hundred posts then it wouldn't be a big deal when you delete the old forum.

Second, you'll eventually have to delete the old forum and that will cause major havoc. Since the old forum with its wealth of knowledge will be there, people in the new forum will be crosslinking to it like crazy. "Oh yeah your problem is just like this thread, check it out." In the meantime, the old forum become more and more dated, possibly exposing security issues. You might be forced to shut down the forum entirely...causing big problems in the new forum because all these links point nowhere.

The existence of two forums will be very confusing to new users. Anyone stumbling into the old forum from a search result will be struck by the tackiness of having a giant banner at the top directing them to the new forum. It the kind of stunt people don't do anymore, really.

I really hope the old forum data can be migrated into the new forum somehow...even an Archive section is much better than the alternative. Of course, when you choose another forum system in a couple years, you start having archives within archives...refactoring the old topics into the new structure is a better overall solution. And yeah, it will take more work than just putting an empty forum up.

I have zero time to contribute to migration beyond my above warnings from past experience, but I hope you guys can figure it out somehow.

I like SMF emoticons/smiley's, but just deleting everything.... ::slight_smile:

Nice to see some constructive discussion on this. Timing could be a bit better but hey, better late than never.

Jeroen

I really hope the old forum data can be migrated into the new forum somehow...even an Archive section is much better than the alternative.

Get it migrated into an archive and then let some people move threads over to the new topics. I'm sure that it would not take too long to get the threads moved manually. I'd say I could easily do 100 a day or so without too much trouble.

Mowcius

Wish i could help you people !
:frowning: i don't know anything related to converting things but
I could help you probably in the dumbest possible ways :wink:
i can just copy paste the members status and post count into a spread sheet ::slight_smile: :-[

i can just copy paste the members status and post count into a spread sheet

I think that would be rather a waste of time :smiley:

I think that would be rather a waste of time Cheesy

I know that ! :slight_smile: but just in case is they would find it helpful i would do it i guess at max a ten hours work ! i could do it for all the wonderful people out here ( including you :wink:

But.....but.....but.....,
SMF already provide a YABB converter!?!