Jump to content

Welcome to CustomsForge Forums
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. This message will be removed once you have signed in.
Login to Account Create an Account

Learn & Play Rocksmith!

Want to play the songs you love? Registration and the use of this website is 100% free.
The only thing you need is a legal copy of Rocksmith 2014 w/ a cable. Click here to buy it.
 
Having trouble logging in or registering? Please click "Support" below.


Announcements:

We need to get half of our donation completed goal asap. Any amount helps click here to donate.

Join us on Discord | Facebook | Twitter | Patreon | Merch | Volunteer | Newsletter Issue 13 (Nov)


Unleashed2k

Unleashed2k

Member Since 17 Feb 2014
OFFLINE Last Active Today, 01:36 AM
*****

CustomsForge November Update | Newsletter (Issue 13)

30 November 2019 - 10:49 PM

To view this months newsletter please use a PDF Reader such as Adobe Acrobat Reader
You can also download this PDF here: http://customsforge....er_nov_2019.pdf


Official Discord

24 October 2019 - 05:10 PM

You can find additional support on the official discord:
 
https://discord.gg/z4UE7vE

CustomsForge October Update | Newsletter (Issue 12) | Interview with Dardo (Newsletter...

23 October 2019 - 09:08 PM

To view this months newsletter please use a PDF Reader such as Adobe Acrobat Reader
You can also download this PDF here: http://customsforge....er_oct_2019.pdf


State of CustomsForge - Full - 10/18/2019

18 October 2019 - 03:25 PM

Thank you for all the continued support for CustomsForge it means the world to us.
I want to start off by apologizing for all the downtime recently. It was never intentional and we’ve been running into several issues over the last week. I’ll start at the beginning:
 
Around Saturday (Oct. 12, 2019) there were user and staff reports of 502 and other types of errors. I investigated and started adjusting the server configs and other files. Unfortunately, this led to more severe issues so we had to revert to our server backups. For our backups we have two servers - one for MySQL and one for data. The MySQL server was reverted back to 7 days due to the fact that the last two backups had problems in the database. 
 
Additionally, we had to run a backup restore on our data server. Due to the mass amount of files on that server, it can take up to 16 hours to complete. Our backups failed multiple times. We had to have our host work on it and help fix issues with the recovery. While that was running (16 hours 5-6 times is a very big pain and super time consuming) we did have a backup that failed but mostly at OS level. I was able to recover a majority of the files and start a donation goal so we could launch a new server while our host tried to fix the backups. Thanks to Chainbrain and others, we were able to get that server going and I was able to bring CustomsForge back online.
 
From there even when the server was recovered we’ve been running into multiple problems with the servers including DNS, SSL, and email issues. I’ve been working around the clock trying to resolve any outstanding issues and we still have a lot of work ahead of us.
 
The Road Ahead: 
  • Fix login issues on the website and improve performance
  • Rewrite some basic functions on Ignition & the forum backend to prep for MySQL migration
  • Migrate MySQL to our new improved web server
  • Prep the website for Forum & Ignition upgrade (please note: no ETA but our goal is within the next few months)
 
Thanks to everyone for the support and once again I apologize for the downtime, at the moment it’s just me working on everything. 
 
If you’re experienced in Laravel, MySQL, PHP, and Server Administration we’re always looking for people to add to the team to help. You can apply here: https://goo.gl/forms/yWt0FX8GQGie3WGG2

State of CustomsForge 10/15/19

15 October 2019 - 12:28 AM

I'll be writing a more in-depth post about what happened and what we are doing in the future to fix the website.

At the moment we are still working with our host to try to recover our backups as they are all failing.

To recap the last few days: The website was having multiple issues all around including random errors, pages not loading, records not showing up, and other various issues. We first tried to optimize the website and run repairs on the website. Unfortunately by fixing these issues we caused new problems to occur such as records not loading at all. After that we tried to recover from multiple backups. The backups failed and one of the major problems is that it took 16 hours to run the backup due to the amount of files the server has. 

 

After asking for donations so we could launch an additional server while we tried to recover the server. We launched a new server and recovered what we could, and after 18+ hours of hard work I was able to get the website running again. Unfortunately there are several issues I am working around the clock to resolve.

 

  • We are aware some pages are not correctly working(fixed)
  • Email is not working (this is a separate issue with Amazon SES that we are working on)(fixed)
  • Any posts/CDLC posted within the last 7 days have been lost. You'll need to edit or recreate whatever changes you have made in the last 7 days. 
  • Issues with images not loading(fixed)
  • Login loop errors

We will be working on getting all of the features back up asap.

 

Thanks for all the support in the last few days, I'm sorry for the downtime but it means a lot that so many people care about this community. 

 

- Unleashed2k

EDIT: View full post here:
http://customsforge....-full-10182019/