Unable to access the iofog controller via iofogctl or rest-api though controller is up and running


(Panyala Surya Kiran Reddy) #1

I have set up a remote control-plane on a cloud VM. I set up an ecn setup with two agents and their respective microservices. Most of the time it works fine as expected.
But sometimes I am encountering connectivity issues with iofog-controller out of nowhere.
Please have a look at this.

when i check the remote vm for controller status

iofog-controller is up and running but i am unable to access controller using iofogctl
Received 500 from POST http://controllerip:51121/api/v3/user/login
{“name”:“SequelizeDatabaseError”,“message”:“SQLITE_FULL: database or disk is full”,“stack”:“SequelizeDatabaseError: SQLITE_FULL: database or disk is full\n at Query.formatError (/opt/iofog/controller/lib/node_modules/iofogcontroller/node_modules/sequelize/lib/dialects/sqlite/query.js:422:16)\n at Query._handleQueryResponse (/opt/iofog/controller/lib/node_modules/iofogcontroller/node_modules/sequelize/lib/dialects/sqlite/query.js:73:18)\n at Statement.afterExecute (/opt/iofog/controller/lib/node_modules/iofogcontroller/node_modules/sequelize/lib/dialects/sqlite/query.js:250:31)\n at Statement.replacement (/opt/iofog/controller/lib/node_modules/iofogcontroller/node_modules/sqlite3/lib/trace.js:19:31)\n at Statement.replacement (/opt/iofog/controller/lib/node_modules/iofogcontroller/node_modules/sqlite3/lib/trace.js:19:31)”}

if I restart my VM, it works normally and after a day or so it again gives the same response. please help me understand why sqlite full error is occuring.


(Serge Radinovich) #2

It is likely that the VM’s storage is at capacity. Could you verify this? E.g. via df -h.

(Panyala Surya Kiran Reddy) #3

Yes, It is to do with VM’s storage. it is 99% occupied when I last checked. Probably after some time it reaches 100% and results in disk full error.
Thanks SergeRadinovich. I will update my VM’s storage.