Difference between revisions of "MySQL fails to start"

From Kolmisoft Wiki
Jump to navigationJump to search
 
(12 intermediate revisions by the same user not shown)
Line 10: Line 10:
   Starting mysqld:                                          [FAILED]
   Starting mysqld:                                          [FAILED]


=Step 1=
=Step 0 (reboot)=
 
Restart the server:
 
reboot
 
Check after restart if mysql is working:
 
  service mysqld status
 
=Step 1 (check free HDD space)=
Check if there is left free space on disk.
Check if there is left free space on disk.
If disk is full, remove unnecessary files and start MySQL service again.
If disk is full, remove unnecessary files and start MySQL service again.
Line 22: Line 32:
  /dev/md1        496M  59M  412M  13% /boot
  /dev/md1        496M  59M  412M  13% /boot


=Step 2=
=Step 2 (check log)=
Check if there is any information in log file /var/log/mysqld.log
Check if there is any information in log file /var/log/mysqld.log


Line 34: Line 44:
* files within /var/lib/mysql - 660
* files within /var/lib/mysql - 660
* and its subdirectories - 700
* and its subdirectories - 700
Check if you set permissions correctly:
find /var/lib/mysql -printf '%m %p\n'


Do again:
Do again:
Line 64: Line 78:
If MySQL still fails to start, check if there are any information in log.
If MySQL still fails to start, check if there are any information in log.


=Step 4=
Try to remove ib_logfile0 and ib_logfile1, but make backups first:
cp -fr /var/lib/mysql/ib_logfile* /home/
rm -rf /var/lib/mysql/ib_logfile*


=Step 4 (error 13 /tmp)=
If you get this error:
Can't create/write to file '/tmp/ib0iEPXa' (Errcode: 13)
InnoDB: Error: unable to create temporary file; errno: 13
[ERROR] Plugin 'InnoDB' init function returned error.
[ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
[ERROR] Unknown/unsupported storage engine: InnoDB
[ERROR] Aborting
Do:
mkdir -p /tmp
chmod 1777 /tmp
Restart mysql, if fails - check log.
=Step 5 (messed up root permissions)=
ERROR 1044 (42000): Access denied for user 'root'@'localhost'
Run:
mysqld_safe --skip-grant-table
And give all privileges to the root.
=Step 6 (crashing)=
Log file looks like this:
/usr/libexec/mysqld() [0x787c3d]
/usr/libexec/mysqld(recv_recover_page+0x34f) [0x7896ff]
/usr/libexec/mysqld(buf_page_io_complete+0x548) [0x74aab8]
/usr/libexec/mysqld(fil_aio_wait+0xfa) [0x762c6a]
/usr/libexec/mysqld() [0x7c8ce0]
/lib64/libpthread.so.0() [0x3386807aa1]
/lib64/libc.so.6(clone+0x6d) [0x33864e8bcd]
Make DB file backups and reinstall or proceed here: https://dev.mysql.com/doc/refman/8.0/en/crashing.html
= Step 7=
Try to remove ib_logfile0 and ib_logfile1, (with backups):
mv -fr /var/lib/mysql/ib_logfile* /home/


And start MySQL again:
And start MySQL again:
Line 75: Line 130:
If MySQL still fails to start, check if there any information in log (/var/log/mysqld.log).
If MySQL still fails to start, check if there any information in log (/var/log/mysqld.log).


=Step 5=
=Step 8=
Run this command to check more information when MySQL starts:
Run this command to check more information when MySQL starts:
  sh -x /etc/init.d/mysqld start
  sh -x /etc/init.d/mysqld start
Line 245: Line 300:
  + exit 1
  + exit 1
  [root@CentOS-69-64-minimal ~]#
  [root@CentOS-69-64-minimal ~]#
<br><br>
= See also =
* https://dev.mysql.com/doc/refman/5.7/en/starting-server-troubleshooting.html
* https://dev.mysql.com/doc/refman/5.5/en/error-creating-innodb.html

Latest revision as of 07:27, 18 September 2018



About

MySQL process fails to start.

Looks like this:

 [root@hostname ~]# service mysqld restart
 Stopping mysqld:                                           [  OK  ]
 MySQL Daemon failed to start.
 Starting mysqld:                                           [FAILED]

Step 0 (reboot)

Restart the server:

reboot

Check after restart if mysql is working:

 service mysqld status

Step 1 (check free HDD space)

Check if there is left free space on disk. If disk is full, remove unnecessary files and start MySQL service again.

Example which shows full (100%) /dev/md2:

[root@hostname]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/md2        204G  204G   16G  100% /
tmpfs            16G     0   16G   0% /dev/shm
/dev/md1        496M   59M  412M  13% /boot

Step 2 (check log)

Check if there is any information in log file /var/log/mysqld.log

If there is no NEW information in this log file, do:

/etc/init.d/mysqld restart

and recheck. If still there is no NEW information in this log file, set such permissions with chmod:

  • /var/lib/mysql - 755
  • files within /var/lib/mysql - 660
  • and its subdirectories - 700

Check if you set permissions correctly:

find /var/lib/mysql -printf '%m %p\n'

Do again:

/etc/init.d/mysqld restart

MySQL can start. If not - check log file again, some new info should appear in it (usually error 13)

Step 3 (error 13)

If you see such error in log:

InnoDB: Operating system error number 13 in a file operation.
InnoDB: The error means mysqld does not have the access rights to
InnoDB: the directory.
InnoDB: File name ./ib_logfile0
InnoDB: File operation call: 'open'.
InnoDB: Cannot continue operation.

Then execute:

chown -R mysql:mysql  /var/lib/mysql

And start MySQL again:

/etc/init.d/mysqld restart

If MySQL still fails to start do:

chmod 777 /var/lib/mysql/*
/etc/init.d/mysqld restart

If MySQL still fails to start, check if there are any information in log.


Step 4 (error 13 /tmp)

If you get this error:

Can't create/write to file '/tmp/ib0iEPXa' (Errcode: 13)
InnoDB: Error: unable to create temporary file; errno: 13
[ERROR] Plugin 'InnoDB' init function returned error.
[ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
[ERROR] Unknown/unsupported storage engine: InnoDB
[ERROR] Aborting

Do:

mkdir -p /tmp
chmod 1777 /tmp

Restart mysql, if fails - check log.

Step 5 (messed up root permissions)

ERROR 1044 (42000): Access denied for user 'root'@'localhost'

Run:

mysqld_safe --skip-grant-table

And give all privileges to the root.

Step 6 (crashing)

Log file looks like this:

/usr/libexec/mysqld() [0x787c3d]
/usr/libexec/mysqld(recv_recover_page+0x34f) [0x7896ff]
/usr/libexec/mysqld(buf_page_io_complete+0x548) [0x74aab8]
/usr/libexec/mysqld(fil_aio_wait+0xfa) [0x762c6a]
/usr/libexec/mysqld() [0x7c8ce0]
/lib64/libpthread.so.0() [0x3386807aa1]
/lib64/libc.so.6(clone+0x6d) [0x33864e8bcd]

Make DB file backups and reinstall or proceed here: https://dev.mysql.com/doc/refman/8.0/en/crashing.html

Step 7

Try to remove ib_logfile0 and ib_logfile1, (with backups):

mv -fr /var/lib/mysql/ib_logfile* /home/

And start MySQL again:

/etc/init.d/mysqld restart

If MySQL still fails to start, check if there any information in log (/var/log/mysqld.log).

Step 8

Run this command to check more information when MySQL starts:

sh -x /etc/init.d/mysqld start

Example:

[root@CentOS-69-64-minimal ~]# sh -x /etc/init.d/mysqld start
+ . /etc/rc.d/init.d/functions
++ TEXTDOMAIN=initscripts
++ umask 022
++ PATH=/sbin:/usr/sbin:/bin:/usr/bin
++ export PATH
++ '[' -z  ']'
++ COLUMNS=80
++ '[' -z  ']'
+++ /sbin/consoletype
++ CONSOLETYPE=pty
++ '[' -f /etc/sysconfig/i18n -a -z  -a -z  ']'
++ . /etc/profile.d/lang.sh
++ unset LANGSH_SOURCED
++ '[' -z  ']'
++ '[' -f /etc/sysconfig/init ']'
++ . /etc/sysconfig/init
+++ BOOTUP=color
+++ RES_COL=60
+++ MOVE_TO_COL='echo -en \033[60G'
+++ SETCOLOR_SUCCESS='echo -en \033[0;32m'
+++ SETCOLOR_FAILURE='echo -en \033[0;31m'
+++ SETCOLOR_WARNING='echo -en \033[0;33m'
+++ SETCOLOR_NORMAL='echo -en \033[0;39m'
+++ PROMPT=yes
+++ AUTOSWAP=no
+++ ACTIVE_CONSOLES='/dev/tty[1-6]'
+++ SINGLE=/sbin/sushell
++ '[' pty = serial ']'
++ __sed_discard_ignored_files='/\(~\|\.bak\|\.orig\|\.rpmnew\|\.rpmorig\|\.rpmsave\)$/d'
+++ cat /proc/cmdline
++ strstr 'ro root=/dev/md/2 rd_NO_LUKS rd_NO_DM nomodeset   consoleblank=0 crashkernel=131M@48M SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=de' rc.debug
++ '[' 'ro root=/dev/md/2 rd_NO_LUKS rd_NO_DM nomodeset   consoleblank=0 crashkernel=131M@48M SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=de' = 'ro root=/dev/md/2 rd_NO_LUKS rd_NO_DM nomodeset   consoleblank=0 crashkernel=131M@48M SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=de' ']'
++ return 1
+ . /etc/sysconfig/network
++ NETWORKING=yes
++ HOSTNAME=CentOS-69-64-minimal
+ exec=/usr/bin/mysqld_safe
+ prog=mysqld
+ STARTTIMEOUT=120
+ STOPTIMEOUT=60
+ MYOPTIONS=
+ '[' -e /etc/sysconfig/mysqld ']'
+ . /etc/sysconfig/mysqld
++ STARTTIMEOUT=120
++ STOPTIMEOUT=60
++ MYOPTIONS=
+ lockfile=/var/lock/subsys/mysqld
+ get_mysql_option mysqld datadir /var/lib/mysql
++ /usr/bin/my_print_defaults mysqld
++ sed -n 's/^--datadir=//p'
++ tail -n 1
+ result=/var/lib/mysql
+ '[' -z /var/lib/mysql ']'
+ datadir=/var/lib/mysql
+ get_mysql_option mysqld socket /var/lib/mysql/mysql.sock
++ /usr/bin/my_print_defaults mysqld
++ sed -n 's/^--socket=//p'
++ tail -n 1
+ result=/var/lib/mysql/mysql.sock
+ '[' -z /var/lib/mysql/mysql.sock ']'
+ socketfile=/var/lib/mysql/mysql.sock
+ get_mysql_option mysqld_safe log-error /var/log/mysqld.log
++ /usr/bin/my_print_defaults mysqld_safe
++ sed -n 's/^--log-error=//p'
++ tail -n 1
+ result=/var/log/mysqld.log
+ '[' -z /var/log/mysqld.log ']'
+ errlogfile=/var/log/mysqld.log
+ get_mysql_option mysqld_safe pid-file /var/run/mysqld/mysqld.pid
++ /usr/bin/my_print_defaults mysqld_safe
++ sed -n 's/^--pid-file=//p'
++ tail -n 1
+ result=/var/run/mysqld/mysqld.pid
+ '[' -z /var/run/mysqld/mysqld.pid ']'
+ mypidfile=/var/run/mysqld/mysqld.pid
+ case "$1" in
+ start
+ '[' -x /usr/bin/mysqld_safe ']'
++ /usr/bin/mysqladmin --socket=/var/lib/mysql/mysql.sock --user=UNKNOWN_MYSQL_USER ping
+ RESPONSE='/usr/bin/mysqladmin: connect to server at '\localhost'\ failed
error: '\Can'\t connect to local MySQL server through socket '\/var/lib/mysql/mysql.sock'\ (2)'\
Check that mysqld is running and that the socket: '\/var/lib/mysql/mysql.sock'\ exists!'
+ '[' 1 = 0 ']'
+ echo '/usr/bin/mysqladmin: connect to server at '\localhost'\ failed
error: '\Can'\t connect to local MySQL server through socket '\/var/lib/mysql/mysql.sock'\ (2)'\
Check that mysqld is running and that the socket: '\/var/lib/mysql/mysql.sock'\ exists!'
+ grep -q 'Access denied for user'
+ touch /var/log/mysqld.log
+ chown mysql:mysql /var/log/mysqld.log
+ chmod 0640 /var/log/mysqld.log
+ '[' -x /sbin/restorecon ']'
+ /sbin/restorecon /var/log/mysqld.log
+ '[' '!' -d /var/lib/mysql/mysql ']'
+ chown mysql:mysql /var/lib/mysql
+ chmod 0755 /var/lib/mysql
+ '['  = sos ']'
+ safe_pid=18744
+ ret=0
+ TIMEOUT=120
+ '[' 120 -gt 0 ']'
+ /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --pid-file=/var/run/mysqld/mysqld.pid --basedir=/usr --user=mysql
++ /usr/bin/mysqladmin --socket=/var/lib/mysql/mysql.sock --user=UNKNOWN_MYSQL_USER ping
+ RESPONSE='/usr/bin/mysqladmin: connect to server at '\localhost'\ failed
error: '\Can'\t connect to local MySQL server through socket '\/var/lib/mysql/mysql.sock'\ (2)'\
Check that mysqld is running and that the socket: '\/var/lib/mysql/mysql.sock'\ exists!'
+ mret=1
+ '[' 1 -eq 0 ']'
+ '[' 1 -ne 1 -a 1 -ne 11 ']'
+ echo '/usr/bin/mysqladmin: connect to server at '\localhost'\ failed
error: '\Can'\t connect to local MySQL server through socket '\/var/lib/mysql/mysql.sock'\ (2)'\
Check that mysqld is running and that the socket: '\/var/lib/mysql/mysql.sock'\ exists!'
+ grep -q 'Access denied for user'
+ /bin/kill -0 18744
+ sleep 1
+ let TIMEOUT=120-1
+ '[' 119 -gt 0 ']'
++ /usr/bin/mysqladmin --socket=/var/lib/mysql/mysql.sock --user=UNKNOWN_MYSQL_USER ping
+ RESPONSE='/usr/bin/mysqladmin: connect to server at '\localhost'\ failed
error: '\Can'\t connect to local MySQL server through socket '\/var/lib/mysql/mysql.sock'\ (2)'\
Check that mysqld is running and that the socket: '\/var/lib/mysql/mysql.sock'\ exists!'
+ mret=1
+ '[' 1 -eq 0 ']'
+ '[' 1 -ne 1 -a 1 -ne 11 ']'
+ echo '/usr/bin/mysqladmin: connect to server at '\localhost'\ failed
error: '\Can'\t connect to local MySQL server through socket '\/var/lib/mysql/mysql.sock'\ (2)'\
Check that mysqld is running and that the socket: '\/var/lib/mysql/mysql.sock'\ exists!'
+ grep -q 'Access denied for user'
+ /bin/kill -0 18744
+ echo 'MySQL Daemon failed to start.'
MySQL Daemon failed to start.
+ ret=1
+ break
+ '[' 119 -eq 0 ']'
+ '[' 1 -eq 0 ']'
+ action 'Starting mysqld: ' /bin/false
+ local STRING rc
+ STRING='Starting mysqld: '
+ echo -n 'Starting mysqld:  '
Starting mysqld:  + shift
+ /bin/false
+ failure 'Starting mysqld: '
+ local rc=1
+ '[' color '!=' verbose -a -z  ']'
+ echo_failure
+ '[' color = color ']'
+ echo -en '\033[60G'
                                                           + echo -n '['
[+ '[' color = color ']'
+ echo -en '\033[0;31m'
+ echo -n FAILED
FAILED+ '[' color = color ']'
+ echo -en '\033[0;39m'
+ echo -n ']'
]+ echo -ne '\r'
+ return 1
+ '[' -x /bin/plymouth ']'
+ /bin/plymouth --details
+ return 1
+ rc=1
+ echo
+ return 1
+ return 1
+ exit 1
[root@CentOS-69-64-minimal ~]#



See also