FORUM MIGRATION SCHEDULED JAN 24th

orbitalair, mowcius:

as for moving the data, it is actually pretty simple, thing is ... if you are willing to rearrange 350664 Posts within 49236 Topics from 37469 registered members into the categories of the new forum, I will be more than happy to pass the task over to you guys.

:wink:

That was obviously a joke. Now seriously, thing is that over the last 5 years this forum has generated a lot of data which is valuable to many, however there is also a lot of the same question asked many many times. We want to have a clean start also bringing the community in acting as moderators to curate the best forum possible.

The information will still be available and you will keep your user name and profile. If there are interesting posts or opened topics you want to keep on working with, you are more than welcome to bring them in. It is our goal to offer a more inclusive system that will also allow for non-english speaking people to have more and better subcategories, etc.

orbitalair:

I know you personally don't care about rebranding the forum to look as the Arduino forum, but there have been others that have asked for the opposite. It is a matter of opinion, as you know.

Sure we could make this on Yabb, but it is a fact, and this has been a complain from people for some time, that Yabb is running on Perl and that makes it very slow for the pace of growth Arduino is experiencing. Just to give you some numbers, our online system registers over 30.000.000 hits per month, with over 600.000 unique IPs, that eats the 8CPUs of our machine pretty fast, so optimizing and moving to a DB based system is crucial.

mowcius:

Regarding the forum system we are moving into, it is a strongly tweaked version of SMF. We made a survey over a group of users and there were no clear preferences for any of the forums under evaluation.

We have implemented our own user profiling system and a menu bar "Google style", that will allow users know where in the site they are, whether they are logged in or not, and whether they have editing rights for a certain page. We have in the pipeline implementing other services, but without making this move, we weren't able of making it the proper way.

Regarding the forum system we are moving into, it is a strongly tweaked version of SMF.

David - please say that whatever you're doing, you're keeping the old smiley emoticons; I mean, I really love the ones here on YaBB - other forums I'm on (I guess my guess of vBulletin was wrong, btw) the emoticons just don't look good. The ones on YaBB are clean, concise, and convey what they need to convey nearly perfectly.

-sigh-

So - how is "number of posts" going to be handled? Will everybody go back to "newbie" status? How will real newbies (ie - those who just bought their first Arduino Uno) know the "old timer god members" (those who have had an Arduino for a while, or those who have an original hand-soldered homebuilt ATMega8 board from way-back-when)?

That would have to be my main concern (waaay above the emoticon issue!)...

:slight_smile:

A lot of EXPECTATIONS as well as DOUBTS ! the only thing i can do is to WAIT AND SEE
and yeah I guess every one out here loves these emoticons :slight_smile:

So - how is "number of posts" going to be handled? Will everybody go back to "newbie" status?

Though that wouldn't mean much to me at least the REAL GOD'S out there should have their status to themselves ( lol i would be even happier if even i have a title God member :wink: )

P.S i never had any idea this was really going to happen

Make an open source community program to transfer all the data.

Perhaps it could be possible to integrate the actual forum in the search engine of the new one? Hope this is "migration" is decided not to have less duplicates, because I think that the same questions are going to be asked forever...

as for moving the data, it is actually pretty simple, thing is ... if you are willing to rearrange 350664 Posts within 49236 Topics from 37469 registered members into the categories of the new forum, I will be more than happy to pass the task over to you guys.

I will do it, that is one of the things a computer can handle, now which forum system are you moving to so I can start to prepare, and when can I expect a copy of the data

not joking (I may not be successful but I will at least try my best, and I have moved a forum nearly as large over 4 systems to get from A to B with a rotten database)

I may not be successful but I will at least try my best, and I have moved a forum nearly as large over 4 systems to get from A to B with a rotten database

Be careful what you ask for, Osgeld... :wink:

Based on the blog posting (and I may be completely wrong about this), there seemed to be the implication that this current forum -isn't- using a DB back-end under Perl (flat text files, anyone?)...

If true, such a system could be a real nightmare to migrate without some serious script-fu to stuff the data from those files into a DB properly. It wouldn't be impossible, but it wouldn't be quick nor easy, either.

Good luck if you manage to pull the task!

:slight_smile:

it is a flat file system, but step 1 is to get it into SMF,which there are scripts available already, from there it can be converted to just about anything
:wink:

as for moving the data, it is actually pretty simple, thing is ... if you are willing to rearrange 350664 Posts within 49236 Topics from 37469 registered members into the categories of the new forum, I will be more than happy to pass the task over to you guys.

Yeah I'm willing to help with this.

So - how is "number of posts" going to be handled? Will everybody go back to "newbie" status? How will real newbies (ie - those who just bought their first Arduino Uno) know the "old timer god members" (those who have had an Arduino for a while, or those who have an original hand-soldered homebuilt ATMega8 board from way-back-when)?

Also a concern.

Presumably post counts could be manually edited in the system. Give everyone the post count they currently have.
Also what about 'member since' - is that going to be reverted? - Please no...

And my private messages?

Good Evening mowcius, you have 274 messages, and no new messages.

What's to happen with them if the system gets changed - can I still access them at least?

Mowcius

I would not really worry about putting things into new catagories, convert, stuff in a read only archive subsection and call it a day, but with conversion you have everything searchable in 1 system and usernames / passwords / private messages are preserved

1.3 cents

I would not really worry about putting things into new catagories, convert, stuff in a read only archive subsection and call it a day

Perhaps it would be possible for a user to transfer a thread when required from the archive - then they choose which subsection to put it under and it then starts a new thread (with the old posts included).

Not sure if that would be possible/useful.

Anyway - conversion is required...

Mowcius

depends on the system, most plain forum system it would involve a moderator action, maybe a request form, on more real things its quite easy but they are much more than just a forum

I think most of the comments you are making are pretty meaningful and I must admit some of them we had probably not given enough importance to.

Let me elaborate this, I will do it as a list, since it will allow for further commenting by those of you interested. I think the discussion at this level is very healthy, and even if I might not be able of sleeping in the incoming weeks, I will do my best to make your concerns come to a good end.

  1. the icons: dude, if you like them, I will make sure they are there, but please first give a try to the new ones :slight_smile:

  2. the current status and amount of messages posted: this is doable, we didn't have this into our list, but I can understand the importance of the status for you guys and I will talk to the team in order to try to make this happen, if not the day planned for the migration, then later. I must say our idea was to promote those of you really active to moderators, but that was of course a given

  3. transferring the previous posts to the new system (this is a longer issue, let me explain it)

Yabb is a file based system, it counts with 4 files per user: userinfo, user logs, user posts, and user messages. 37.000 users means 4 times more files ... IN A SINGLE FOLDER

The converters existing for moving from Yabb to SMF are NOT working, sorry for the SMF developers, but it took me a week to hack it and make it work again.

Right now we have stream-lined the process of migrating the users but not so much the posts. Mostly because we consider the new forum has to be structured in a completely different way. After 5 years with the current set of categories, we have gained quite some experience in the topics that matter the most to people and have identified the categories that seem to make the most sense.

The time it takes to migrate the 37.000 users to SMF is irrelevant, in my 2.3GHZ laptop I need about 3 hours. Then the database has to be filtered, because as explained, this is not just about the forum, this is about having a so-called Single Sign On system that will work all over the Arduino system: A single usernam and password to work on the forum, the wiki, the blog (the day we open it for external editors), other wikis, the store, and future services we have in our roadmap.

This means, our SMF is not really SMF, it is SMF made into our own identification system.

However, even if I think it is a perfectly valid point the one of having a single location where to search for information from the forum, we didn't conceive it like that in the first place. One possible solution, that cannot be scheduled for next Monday, but could be in a week more, is to make a category called "old" where to push in all the old messages into a single table, having the current categories as subcategories of that old forum. We could make it so that that table would not be editable but by moderators. Then users could request to move some of those topics to the new boards.

This is a lot of work, but if there are 15 volunteers, we could consider it. To me it will me I will sleep one week less this year, but it will be worth the effort.

Summarizing, I think the following things have to be considered:

  • moving the current user status to the new system
  • adding the current messages as a subcategory just to have them searchable
  • checking the emoticons and maybe use the current ones (just to please you above all the other 36.999 users :slight_smile: )

Something to think about:

  • letting a bunch of moderators access to a read-only archive to -upon request- move topics of importance to the new boards

Any other thoughts?

  • moving the current user status to the new system

Sounds good.

  • adding the current messages as a subcategory just to have them searchable

That sounds like a much better way to have it.

  • checking the emoticons and maybe use the current ones (just to please you above all the other 36.999 users Smiley )

Hehe - I see what he means - can't remember what the SMF ones are like atm but perhaps that is a discussion to have on a thread when the new forum turns up.

We could make it so that that table would not be editable but by moderators. Then users could request to move some of those topics to the new boards.

  • letting a bunch of moderators access to a read-only archive to -upon request- move topics of importance to the new boards

Yes - something like that sounds like it could be an acceptable way to go - not sure how 'requests' would be best submitted. Perhaps we can have a 'request' button on the 'archive' forum section.

Any other thoughts?

Private messages? Can they be stored at least or some way of backing them up - I won't have the time to try and back them up individually before a forum switch (and that would be one hell of a copy/paste job).

Maximum dimensions on images (preferably with auto-resize) so we don't get people posting a few thousand px images slowing it all down and making the threads look a mess. Thumbnails for images over ~600px wide is possibly the best way to go here.

This is a lot of work, but if there are 15 volunteers, we could consider it.

Count me in for helping.

Mowcius

not to be blunt but if your thinking of a single sign on system why not use a single system (ie joomla) to handle it all instead of patchworking all these little systems together through bridges and hacks which makes even a simple future upgrade (ie security) a nightmare as you fight with rehacking stuff and opps bridge 1.2.1 doesnt work with front end 1.1.2 and etc

count me in on helping too (if that were not evident)

(and for the record I have had the smf converter working for yabb 2.2.3 for a few months now, whenever it was you put up that poll / test forum :wink: though its not perfect, its never perfect)

What about a preview of what the new forum is going to look like?

It is pretty clear everyone has hopes and wishes for whatever is coming next. Not everyone is going to be happy. But it would take a lot of guesswork (and apparently worrying) if the interested parties could see what the new forum looks like.

May I suggest more local boards, like USA/MN so we can organize local gatherings to share experience face to face. Helps to know who does what in your area, don't you think so?

I've seen "migration" attempts where one forum was left frozen as an archive. It's horrible.

First of all, the huge number of posts in an existing forum should be taken as MORE of an incentive to move that data into the new forum. The more posts you have, the greater the value of that data set. If you had only a few hundred posts then it wouldn't be a big deal when you delete the old forum.

Second, you'll eventually have to delete the old forum and that will cause major havoc. Since the old forum with its wealth of knowledge will be there, people in the new forum will be crosslinking to it like crazy. "Oh yeah your problem is just like this thread, check it out." In the meantime, the old forum become more and more dated, possibly exposing security issues. You might be forced to shut down the forum entirely...causing big problems in the new forum because all these links point nowhere.

The existence of two forums will be very confusing to new users. Anyone stumbling into the old forum from a search result will be struck by the tackiness of having a giant banner at the top directing them to the new forum. It the kind of stunt people don't do anymore, really.

I really hope the old forum data can be migrated into the new forum somehow...even an Archive section is much better than the alternative. Of course, when you choose another forum system in a couple years, you start having archives within archives...refactoring the old topics into the new structure is a better overall solution. And yeah, it will take more work than just putting an empty forum up.

I have zero time to contribute to migration beyond my above warnings from past experience, but I hope you guys can figure it out somehow.

I like SMF emoticons/smiley's, but just deleting everything.... ::slight_smile:

Nice to see some constructive discussion on this. Timing could be a bit better but hey, better late than never.

Jeroen