New SBFspot Installation Queries

bobboulby · June 20, 2018, 3:25pm

Hi SBF, I wonder if you can help me with the following for a new installation on a Raspberry Pi. I’ve just worked my way through the “Installation Linux SQLite” instructions on your Wiki and have a couple of comments/questions:

I originally installed SBFspot in 2016 and notice that there have been a few minor changes since then. I now see that there are two cron jobs, the 5 minute daydata job is the same as I had running last time but I was wondering what the daily monthdata job does?

I normally change the start and end times for the daydata job as the days lengthen/shorten so do I need to change the time of the monthdata job to always run 5 minutes before the first daydata job?

In the instructions for editing SBFspotUpload.cfg you say “LogDir : Path for logfiles /var/log/sbfspot.3” however further up the file it is referenced as LogDir=/home/pi/smadata/logs which is what it was in the 2016 version. Which is correct please?

I was getting errors when I executed “sudo systemctl start SBFspotUpload” due to a missing /home/pi/smadata/logs directory. Creation of this directory should be added to the instructions.

Thanks - Bob.

bobboulby · June 20, 2018, 3:52pm

Sorry one more question. Following my post above, everything was working ok until I restarted my Pi. I now get:

sudo systemctl status SBFspotUpload
● SBFspotUpload.service - SBFspot Upload Daemon
Loaded: loaded (/usr/local/bin/sbfspot.3/SBFspotUpload.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2018-06-20 16:43:38 BST; 49s ago
Process: 402 ExecStart=/usr/local/bin/sbfspot.3/SBFspotUploadDaemon -c /usr/local/bin/sbfspot.3/SBFspotUpload.cfg -p /

Jun 20 16:43:38 raspberrypi systemd[1]: Starting SBFspot Upload Daemon…
Jun 20 16:43:38 raspberrypi systemd[1]: SBFspotUpload.service: Control process exited, code=exited status=1
Jun 20 16:43:38 raspberrypi systemd[1]: Failed to start SBFspot Upload Daemon.
Jun 20 16:43:38 raspberrypi systemd[1]: SBFspotUpload.service: Unit entered failed state.
Jun 20 16:43:38 raspberrypi systemd[1]: SBFspotUpload.service: Failed with result ‘exit-code’.

and the log file says:

[16:43:38] INFO: Starting SBFspotUploadDaemon Version 1.1.0
[16:43:38] ERROR: Could not open pid file /run/user/1000/sbfspotupload.pid : No such file or directory

The folder exists but not the .pid file:

pi@raspberrypi:/run/user/1000 $ ls -l
total 0
srw-rw-rw- 1 pi pi 0 Jun 20 16:43 bus
drwx------ 2 pi pi 120 Jun 20 16:43 gnupg
dr-x------ 2 pi pi 0 Jun 20 16:43 gvfs
srwxr-xr-x 1 pi pi 0 Jun 20 16:43 menu-cached-:0
srwxr-xr-x 1 pi pi 0 Jun 20 16:43 pcmanfm-socket–0
drwxr-xr-x 3 pi pi 100 Jun 20 16:43 systemd

Thanks.

SBF · June 21, 2018, 8:33pm

The 5min daydata job has the -am0 parameter to avoid getting unchanged monthdata every 5 minutes.
The second job gets only the monthdata with the new day.

No, this is not needed. But changing start/end would be something nice to automate.

Both are OK, but I tend to use the Linux Filesystem Hierarchy Standard where/when I can. So for new installations, /var/log/sbfspot.3 is the preferred directory for logfiles. I’ll have to correct the wiki if the two are used.

Like already said, it’s better to use /var/log/sbfspot.3

Hope this helps

SBF · June 21, 2018, 8:38pm

Is this OK? There is no PID file in the ExecStart (or is it just truncated by systemctl?)

bobboulby · June 22, 2018, 6:42am

Yes it was truncated. It actually says:

pi@raspberrypi:~ $ sudo systemctl status SBFspotUpload
● SBFspotUpload.service - SBFspot Upload Daemon
Loaded: loaded (/usr/local/bin/sbfspot.3/SBFspotUpload.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2018-06-22 06:36:45 BST; 35s ago
Process: 390 ExecStart=/usr/local/bin/sbfspot.3/SBFspotUploadDaemon -c /usr/local/bin/sbfspot.3/SBFspotUpload.cfg -p /run/user/1000/sbfspotupload.pid 2>&1> /dev/null (code=exited, status=1/FAILURE)

Jun 22 06:36:45 raspberrypi systemd[1]: Starting SBFspot Upload Daemon…
Jun 22 06:36:45 raspberrypi systemd[1]: SBFspotUpload.service: Control process exited, code=exited status=1
Jun 22 06:36:45 raspberrypi systemd[1]: Failed to start SBFspot Upload Daemon.
Jun 22 06:36:45 raspberrypi systemd[1]: SBFspotUpload.service: Unit entered failed state.
Jun 22 06:36:45 raspberrypi systemd[1]: SBFspotUpload.service: Failed with result ‘exit-code’.

/usr/local/bin/sbfspot.3/SBFspotUpload.service contains:

pi@raspberrypi:~/smadata/logs $ cat /usr/local/bin/sbfspot.3/SBFspotUpload.service
[Unit]
Description=SBFspot Upload Daemon
After=network.target

[Service]
User=pi
Type=forking
TimeoutStopSec=60
ExecStart=/usr/local/bin/sbfspot.3/SBFspotUploadDaemon -c /usr/local/bin/sbfspot.3/SBFspotUpload.cfg -p /run/user/1000/sbfspotupload.pid 2>&1> /dev/null
PIDFile=/run/user/1000/sbfspotupload.pid
Restart=no
RestartSec=60
SuccessExitStatus=SIGKILL

[Install]
WantedBy=default.target

Bob.

bobboulby · June 22, 2018, 5:56am

Sorry another question. I’d like to suppress the generation of all the .csv files including events. Am I right in thinking that I just set CSV_Export=0 in SBFspot.cfg?

Bob.

bobboulby · June 22, 2018, 1:23pm

Sorry this is a long post!

Tried a few things today and I found that if I start the SBFspot Upload Daemon from the command line everything works however after a reboot the Daemon doesn’t restart automatically. Details below:

pi@raspberrypi:/usr/local/bin/sbfspot.3 $ sudo systemctl start SBFspotUpload

pi@raspberrypi:/usr/local/bin/sbfspot.3 $ sudo systemctl status SBFspotUpload
● SBFspotUpload.service - SBFspot Upload Daemon
Loaded: loaded (/usr/local/bin/sbfspot.3/SBFspotUpload.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2018-06-22 12:54:09 BST; 29s ago
Process: 5630 ExecStart=/usr/local/bin/sbfspot.3/SBFspotUploadDaemon -c /usr/local/bin/sbfspot.3/SBFspotUpload.cfg -p
Main PID: 5631 (SBFspotUploadDa)
CGroup: /system.slice/SBFspotUpload.service
└─5631 /usr/local/bin/sbfspot.3/SBFspotUploadDaemon -c /usr/local/bin/sbfspot.3/SBFspotUpload.cfg -p /run/use

Jun 22 12:54:09 raspberrypi systemd[1]: Starting SBFspot Upload Daemon…
Jun 22 12:54:09 raspberrypi systemd[1]: Started SBFspot Upload Daemon.

pi@raspberrypi:/usr/local/bin/sbfspot.3 $ ls /run/user/1000
bus gnupg gvfs menu-cached-:0 pcmanfm-socket–0 sbfspotupload.pid systemd

pi@raspberrypi:/usr/local/bin/sbfspot.3 $ cat /run/user/1000/sbfspotupload.pid
5631

So pid file exists

pi@raspberrypi:~/smadata/logs $ cat SBFspotUpload20180622.log
[06:36:45] INFO: Starting SBFspotUploadDaemon Version 1.1.0
[06:36:45] ERROR: Could not open pid file /run/user/1000/sbfspotupload.pid : No such file or directory
[12:54:09] INFO: Starting SBFspotUploadDaemon Version 1.1.0
[12:54:09] INFO: Starting Daemon…
[12:54:11] INFO: Uploading 100 datapoints, starting with 20180620,16:35,837435,744,283.48,45.7,6.571 => OK (200)
[12:55:30] INFO: Uploading 100 datapoints, starting with 20180621,09:15,841327,1116,287.14,32.8,5.139 => OK (200)
[12:56:31] INFO: Uploading 100 datapoints, starting with 20180621,17:35,852163,912,282.95,42.8,4.277 => OK (200)
[12:57:30] INFO: Uploading 31 datapoints, starting with 20180622,10:25,856598,1596,266.53,44.4,6.376 => OK (200)
[13:00:30] INFO: Uploading datapoint: 20180622,13:00,861118,1812,253.8,53.9,7.621 => OK (200)

So working

Following shutdown and restart …

pi@raspberrypi:~ $ sudo systemctl status SBFspotUpload
● SBFspotUpload.service - SBFspot Upload Daemon
Loaded: loaded (/usr/local/bin/sbfspot.3/SBFspotUpload.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2018-06-22 13:06:28 BST; 1min 32s ago
Process: 386 ExecStart=/usr/local/bin/sbfspot.3/SBFspotUploadDaemon -c /usr/local/bin/sbfspot.3/SBFspotUpload.cfg -p /

Jun 22 13:06:28 raspberrypi systemd[1]: Starting SBFspot Upload Daemon…
Jun 22 13:06:28 raspberrypi systemd[1]: SBFspotUpload.service: Control process exited, code=exited status=1
Jun 22 13:06:28 raspberrypi systemd[1]: Failed to start SBFspot Upload Daemon.
Jun 22 13:06:28 raspberrypi systemd[1]: SBFspotUpload.service: Unit entered failed state.
Jun 22 13:06:28 raspberrypi systemd[1]: SBFspotUpload.service: Failed with result ‘exit-code’.

pi@raspberrypi:~ $ ls /run/user/1000
bus gnupg gvfs menu-cached-:0 pcmanfm-socket–0 systemd

So Daemon not started and no pid file

pi@raspberrypi:~ $ sudo systemctl start SBFspotUpload
pi@raspberrypi:~ $ sudo systemctl status SBFspotUpload
● SBFspotUpload.service - SBFspot Upload Daemon
Loaded: loaded (/usr/local/bin/sbfspot.3/SBFspotUpload.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2018-06-22 13:10:54 BST; 6s ago
Process: 945 ExecStart=/usr/local/bin/sbfspot.3/SBFspotUploadDaemon -c /usr/local/bin/sbfspot.3/SBFspotUpload.cfg -p /
Main PID: 946 (SBFspotUploadDa)
CGroup: /system.slice/SBFspotUpload.service
└─946 /usr/local/bin/sbfspot.3/SBFspotUploadDaemon -c /usr/local/bin/sbfspot.3/SBFspotUpload.cfg -p /run/user

Jun 22 13:10:54 raspberrypi systemd[1]: Starting SBFspot Upload Daemon…
Jun 22 13:10:54 raspberrypi systemd[1]: Started SBFspot Upload Daemon.

pi@raspberrypi:~ $ ls /run/user/1000
bus gnupg gvfs menu-cached-:0 pcmanfm-socket–0 sbfspotupload.pid systemd

Working again

Then I tried re-enabling the service but it still didn’t survive a reboot.

pi@raspberrypi:~ $ sudo systemctl enable /usr/local/bin/sbfspot.3/SBFspotUpload.service

Shutdown and restart

pi@raspberrypi:~ $ sudo systemctl status SBFspotUpload
● SBFspotUpload.service - SBFspot Upload Daemon
Loaded: loaded (/usr/local/bin/sbfspot.3/SBFspotUpload.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2018-06-22 13:48:13 BST; 2min 36s ago
Process: 402 ExecStart=/usr/local/bin/sbfspot.3/SBFspotUploadDaemon -c /usr/local/bin/sbfspot.3/SBFspotUpload.cfg -p /

Jun 22 13:48:13 raspberrypi systemd[1]: Starting SBFspot Upload Daemon…
Jun 22 13:48:13 raspberrypi systemd[1]: SBFspotUpload.service: Control process exited, code=exited status=1
Jun 22 13:48:13 raspberrypi systemd[1]: Failed to start SBFspot Upload Daemon.
Jun 22 13:48:13 raspberrypi systemd[1]: SBFspotUpload.service: Unit entered failed state.
Jun 22 13:48:13 raspberrypi systemd[1]: SBFspotUpload.service: Failed with result ‘exit-code’.

Manual start - working again

pi@raspberrypi:~ $ sudo systemctl start SBFspotUpload
pi@raspberrypi:~ $ sudo systemctl status SBFspotUpload
● SBFspotUpload.service - SBFspot Upload Daemon
Loaded: loaded (/usr/local/bin/sbfspot.3/SBFspotUpload.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2018-06-22 13:51:17 BST; 4s ago
Process: 912 ExecStart=/usr/local/bin/sbfspot.3/SBFspotUploadDaemon -c /usr/local/bin/sbfspot.3/SBFspotUpload.cfg -p /
Main PID: 913 (SBFspotUploadDa)
CGroup: /system.slice/SBFspotUpload.service
└─913 /usr/local/bin/sbfspot.3/SBFspotUploadDaemon -c /usr/local/bin/sbfspot.3/SBFspotUpload.cfg -p /run/user

Jun 22 13:51:17 raspberrypi systemd[1]: Starting SBFspot Upload Daemon…
Jun 22 13:51:17 raspberrypi systemd[1]: Started SBFspot Upload Daemon.

I’m a bit out of my depth here, really appreciate your help.

SBF · June 22, 2018, 3:20pm

Yes, or adding -nocsv to the command line

SBF · June 23, 2018, 9:02pm

I was able to reproduce the issue. I’ll come back later when I have a fix.

bobboulby · June 24, 2018, 5:38am

Thanks SBF appreciated but before you spend any more time on it, I thought that I"d try installing everything again from scratch. Maybe I made a mistake somewhere when setting things up originally. Probably won’t have chance for a few days though.

Bob.

SBF · June 24, 2018, 6:44am

No, reinstalling makes no sense.
It seems /run/user is not yet available when the daemon starts. For now you can set the restart option:

Restart=on-failure
RestartSec=15s

You might see a few “pid not found” in the log, but daemon will run.

It’s a workaround, so I’ll look a bit further to find another solution.

bobboulby · June 24, 2018, 3:49pm

I made the change you suggested (without the “s” in RestartSec=15s") and the Daemon now starts on reboot . As you said, there is still an error in the log file though:

[16:43:11] INFO: Stopping Daemon…
[16:43:16] INFO: Starting SBFspotUploadDaemon Version 1.1.0
[16:43:16] ERROR: Could not open pid file /run/user/1000/sbfspotupload.pid : No such file or directory
[16:43:32] INFO: Starting SBFspotUploadDaemon Version 1.1.0
[16:43:32] INFO: Starting Daemon…
[16:45:30] INFO: Uploading datapoint: 20180624,16:45,893211,1128,261.19,51,4.531 => OK (200)

Thanks again - Bob.

mi_me · July 2, 2018, 5:19pm

Hello,
Thanks for your fine support. I had the same problem … the service is starting after reboot, but no pid-file was found.
Mike

latspt · July 18, 2018, 11:06am

Hi all,

I had been using a Windows 10 laptop with PV Bean Counter to send data to PVOutput, but a Windows ‘update’ caused the Bluetooth connection to my SMA inverter to stop working, so I decided to trade up to a Raspberry Pi.

So, I bought a Raspberry Pi 3 Model B+ and after some research, decided to use SBFspot to send the data to PVOutput, with one slight problem - I have zero knowledge of Linux.

I found the instructions at https://github.com/SBFspot/SBFspot/wiki to set it up, so started to give it a go (using the Version 3.4.0 and higher: Installation Linux SQLite instructions)
Everything seemed to go fine until I reached the “Install Curl” section when things started going rogue. (see the screenshot below)
curl%20issues

Any ideas from the brains trust?

Thanks in advance
Lindsay

SBF · July 18, 2018, 12:06pm

Sorry, can’t read anything on screenshot. Can you copy/paste it here (between code tags)

latspt · July 18, 2018, 12:11pm

Sorry, I’ve attached a larger filesize of the screenshot (took a photo as I was on a separate computer).
Hopefully this is better

cheers
Lindsay

bankstownbloke · July 18, 2018, 1:19pm

Looks like a DNS issue, the pi is unable to reach the hostname http://raspbian.raspberrypi.org used to install curl

https://www.raspberrypi.org/forums/viewtopic.php?t=17016#p171805

latspt · July 19, 2018, 12:42pm

Superstar - I was operating from wifi. Plugged in the ethernet cable and tried again and it worked fine.
Thanks bankstownbloke!

latspt · September 1, 2018, 3:14am

Ok, so I’m back again! I progressed a bit further with the installation, but ran into a few issues - so it has sat there for a while. The main one is that when I start the upload daemon it seems to only upload one day’s worth of output and then stops. I think I’ve made a couple of missteps in the setup process and want to start from scratch.

My RPi is a Model 3 B+ and came pre-installed with everything setup and running. Is there an easy way I can get it back to that state (ie. wipe everything I’ve done since then) so I can start fresh?

Screenshot of the error messages I’m getting just in case there’s an easy fix. I’d be happy to supply any further info to help troubleshoot, but not sure what would help.

Thanks in advance

SBF · September 1, 2018, 8:39pm

Is there something in SBFspotUpload.log?
Try to add RestartSec=10 to the [Service] section