[Solved] Raspi 3 eOS 2.5.6 issues?

My eBlocker Pi is working fine so far until it becomes unaccessible after a while. Then I reboot it via a USB-connected keyboard using ctrl-alt-entf. Do you have any idea what to try?

Raspberry Pi 3
RAM 4MB? How can I find out?
SD Card Sandisk 64 GB A2
Power Supply 5V 2A

eOS 2.5.6

@serpentin

please read the intructions!

Do really have a powersupply with 2A ?

You have to use one with 2,5A 🤨 

with less power eBlocker could crash 💣 

Regards

PIO78

@serpentin @pio78 We suspect a low memory condition caused by filter list updates are the reason for the Raspi 3 issues. Thanks to @bpr for this finding during intensive tests for the 2.5.8 update. He also found a clever way how to save memory by shrinking the malware list without compromising safety. 😎 🤩 😎 

We hope to release the improved lists as well as 2.5.8+ during the week. 

@serpentin Please stay tuned and share your experience after the update here.

Same issue here.

eBlocker stops working random... after 2h, 3h, 1,5h...

eBlocker Webinterface is no longer reachable, can't even ping the device. Ethernet LEDs are still flashing.

Set it up yesterday for the first time and sadly this is my first experience with eblocker, but i love it. i was using pihole before.

My setup:

• Model: Raspberry Pi 3 Model B Plus Rev 1.3
• RAM: 1GB
• SD Card: 16 GB Transcend SDHC Class 10
• PowerSupply: 5V 2,3 A
• eBlocker version 2.5.6

My first steps after installation:

• I enabled eBlocker mobile after installation but havent used it yet.
• Renamed all my devices in eblocker webinterface (~32) and enabled them to use eblocker.
• Added 2 more dns servers and another Filterlist (oisd domain).

Then i ran into this issue.

First time i didn't complain. Just powered off hard and restarted it. it just can happen...

After that it worked as expected again.

then, after a period of time (about 2 to 3 hours) it stopped working again.

I decided to undo my following settings:

• disabled eblocker mobile
• removed added dns servers and oisd filter list.

unfortunately it stops working again after some time.

hope my expierinces can help you debugging.

@safri THX for the helpful feedback.

Please disable additional blocker lists as memory consumption of anything „additional“ seems to be the cause.

Last night we‘ve updated the eBlocker filters in a less memory consuming matter. This should (hopefully) fix the issues.

If you are not experiencing issues with the eBlocker standard for a couple of days then you might want to add additional list or enable eBlocker VPN (which is probably another source of more memory consumption).

THX very much for your cooperation!

short feedback:

1. i replaced the powersupply & cable to an recommended 5V 2,5A 

2. Reinstalled eblocker, run through setup wizard and only activated my ~30 devices, so no further changes done. Just to test out of the box configuration.

3. updated eblocker to lastest available

Unfortenatly same behavior like before. It works perfectly for 30 mins to 2 h and then suddenly the raspi stops working.

cant even ping the device anymore. just hard reset and then it is working again for the next few hours.

i decided to enable root access on eblocker to watch the system via ssh.

i couldnt find a ram problem while watching with htop. average RAM usage was ~700 of max 926 MB unitl ssh connection froze.

i generated a diagnostic report after i rebooted the device.

furthermore, i could find any suspicious entries in any log files which could be dedicated to this behavior, maybe u can find the issues with the report i send to support(at)eBlocker.org .

extended infos:

My setup:

• Model: Raspberry Pi 3 Model B Plus Rev 1.3
• RAM: 1GB
• SD Card: 16 GB Transcend SDHC Class 10
• PowerSupply: 5V 2,5 A
• eBlocker version 2.5.6
• last update 11/28/20
• Fritzbox 7590
• eblocker automatic mode

@safri Thanks very much for your contribution!

As far as I know some Java-heap space is running out of memory - not the OS. But I'm not a dev anymore...

Our issue is: all Raspi 3 devices used by team members run stable. So it's difficult to analyze if we can't see the error happening. 🤔 

Hope we find something helpful in your logs...

Thanks again!

Thank you for the fresh 2.5.8 update which installed itself automatically.
Unfortunately the error still exists.
Apparently the system freezes if a certain traffic load occurs, but not during idle times, e.g. at night. (my impression. not 100% sure, maybe 67%…)

My system:

eOS 2.5.8
Raspberry Pi 3 B
64 GB A2 Sandisk SD Card
2.5A 5V Power Supply

@serpentin THX for the feedback.

Have you added individual blocking lists or are you using VPN? Anything that might use extra memory..?

THX!

Hi,

Thank you @Random,

No, nothing like that. I only activated https support and device cloaking

@safri @serpentin @pio78 We currently suspect the power supply being the cause of issues.

If you have the chance of changing the power supply to 3A (or more) it would be great if you could retest and confirm. BTW: 2.5A is not enough (especially for the test to confirm)

THX!

This suspicion also sprouted in me.
I have been using a 3A power supply for a few hours. Unfortunately the Pi has got stuck again 2 times in the meantime. (.... means unresponsive, it usually does not crash completely)

I ll give it a try.

Especially to proof @serpentin experience and to give u a 2nd feedback about this problem.

Luckily i found a power supply with 5V 3A from an old camera.

EDIT:

does a further diagnostic report help you with the new power supply?

@safri @serpentin 🤔  If 3A+ still fails: Yes, please send the diagnosis report to support (at) eBlocker.org after eBlocker freezes and you have rebooted.

THX!

@Random ok, i will do

So i completly reinstalled eBlocker.

I wanted a clean installation, especially to help you debugging.

I will do a "live logging" for every step i do.

My Setup:

Model: Raspberry Pi 3 Model B Plus Rev 1.3
RAM: 1GB
SD Card: 16 GB Transcend SDHC Class 10
PowerSupply: 5V 3 A

Router:
Fritzbox 7590 (FritzOS 7.21)

Network:
~ 30 Devices
All devices are configured by DHCP, except RasperryPi (eBlocker). It has a reservation for an IP address.

First steps:

Flashing SD Card:

1. Downloaded eBlocker again (12/02/20 7pm v2.5.6) - just to get sure data is actual - FYI in your DIY site the 2.5.6 image is still linked (it should be 2.5.8?).
2. Flashed succesfully with Etcher (v 1.5.111) to SD card
3. Ejected SD Card from Windows securely

no issues so far...

4. Installed SD Card into Rasperry
5. Booted System

Setup Wizard:

6. Language English
7. Accepted License
8. Timzone Europe/Berlin
9. Device Name set "my-eBlocker"
10. "Do not automatically enable new devices (recommended)"
11. Key "FAMLFT-OPENSOURCE" + My E-Mail
12. eBlocker wanted an Update.. accepted and done.
13. Waited ~ 10 minutes until eblocker will restart

PROBLEM: eblocker didn't restart after 1h, so i unplugged for ~ 1 minute and then replugged power

14. eBlocker booted. Release notes appeared, accepted and started.
15. Alle devices showed up deactivated.
16. Checked if block lists are active
17. checked if eblocker is activated as DNS server
18. checked custom DNS servers are available
19. checked version of eBlocker - was still 2.5.6

Update check manual:

20. On License & Update i started "Check Now" to make sure the newest version is running
21. eblocker told me, there were 18 Updates available
22. Clicked "Update Now" and confirmed pop up message
23. Update running

PROBLEM: eblocker didn't restart - same as before. i was waiting for 25 minutes. so i unplugged for ~ 1 minute and then replugged power [maybe here is a problem in the update process, i got this several times. the device will not reboot itself]

24. eBlocker booted
25. checked OS Version, still 2.5.6
26. 2nd time to check manual update now to get hopefully OS Version 2.5.8
27. still telling me 18 updates available
28. 2nd run "Update Now", confirmed pop up message
29. update running

Update done.

30. This time, device rebooted itself after update and now version 2.5.8 is running.
31. Checked if DNS Firewall and 3 default Blocker lists are still enabled.

Enabled Devices:

32. On "Devices" Tab i enabled all ~ 30 devices

Test on iPhone:

33. Checked function on several websites - blocker is working
34. Checked device dashboard and ran succesfully all tests

Test on PC:

35. Checked function on several websites - blocker is working
36. Checked device dashboard and ran succesfully all tests

No more changes done so far.
Hope it works now. I will give feedback tomorrow.

Unluckily this morning the eblocker was not responsive again.

But it seems that the device is still running and only network interface is unresponsive.

Yesterday evening i went to bed and let it run. This morning i had to hard reset it again, but in the "Event" tab the following is shown:

Dec 3rd, 2020 7:02:43 am
eBlocker's main process started.

Dec 3rd, 2020 7:01:43 am
eBlocker's power supply was disconnected.

Dec 2nd, 2020 10:51:03 pm
eBlocker's main process started.

My theorie is, only the ethernet gets shutted down...

I will try another setup now, i switched to "Expert Mode" and will test if there is trouble with my network (or the Fritzbox). So i assigned manually eblocker as GW and DNS to two devices (iphone and pc).

just to make sure that it's not because of the automatic mode and the amount of devices.

i really want to get it work. i like this project.

BTW: eBlocker is now running 2.5.8, so it is not only 2.5.6 affected.

@safri THX for your support!

Posted by: @safri

FYI in your DIY site the 2.5.6 image is still linked (it should be 2.5.8?)

That‘s intended. We are not releasing an image for minor fixes. Image should auto update afterwards.

Posted by: @safri

PROBLEM: eblocker didn't restart after 1h, so i unplugged

Strange. There is a know (and already fixed) bug in 2.5.6 that the progress is not increasing during download. If you have limited bandwidth it might show 0% for quite a while (much longer than 10min). So patience is key... (or it‘s a undiscovered bug).

You didn‘t talk about https / cert install and I wonder if you‘ve enabled it?

THX!

@random

the restart problem after device update occurred several times. it took forever, and at one point i am not willing to wait longer. i waited every time minimum 20 minutes before hard reset.

i tried 3 different SD cards just to make sure this is not the problem.

and every installation i had trouble with the automatic reboot when update is finished.

yesterday it looked like the device needed 2 times to succesfully complete the 2.5.8 update.

my isp is Telekom, 100 VDSL, so speed should not be a problem.

i didnt enabled https/certs.

i wanted to, but the problem with the frozen raspi occurred before. so i dont want to make things more complicated to debug.

if the device is running stable, for sure i will use this function.

btw, since today 7:15:18 am eblocker is running smooth so far. (with 2 devices and expert mode like mentioned in my post above - longest timespan until now - with active devices producing traffic)

Okay, i really thought it was working now... but sadly it didn't. 😥 

It seems this behavior occurs when there is lots of traffic going through the device. These are not founded facts, but I was able to provoke it that way. Sometimes faster, sometimes slower.

I was starting youtube videos and ran some speed-tests.

What did i do?

I enabled root access on eblocker to see what happens when the device stops working. I connected a screen and a keyboard.

I can confirm now that the Raspberry(eblocker) is still running, only the network interface is not responding.

When this issue happened i was quickly checking /var/log/syslog

See appended image.

14:26 was the exact time when device got unresponsive.

I was looking to several other logs, but nothing related found.

I tried to restart network service, but without success.

/etc/init.d/network status tells everything is fine.

I also tried to reinitialize all usb devices with "usbreset" (as far as i know the ethernet on raspi 3 is connected via usb) - unsuccessfully.

i am thinking right now that the raspberry has a hardware problem

I sent a diagnostic report... maybe it helps...

@safri  + @random

Same happend to me with Raspi 3+:

After V2.09 ran happily for months - (and stopped more than 2.5 Mio. tracker-calls and ads)
- I changed to 2.50 > 2.56 > 2.58 and since then it only takes hours, before eblocker becomes unavailable.

So far I changed and tried:
- power-supply (up to 3.5 A)
- cooling to a massive alu-case case
- several sd-cards (8-32GB/sev. brands)
- exchanged the PI3+ completely

- reduced amount of obeserved devices
- no extra functions activated

Just bought the second Pi3+-replacement, when I found this post.
There seems to be really the same problem with this Pi;

Sadly I dont have enough time , to dive in so deeply as safri.
I just hope a fix will come soon ...

Greetings from Cologne

I want to use eblocker as quickly as possible, so i decided to buy a Pi4 4gb.

Installed, configured, works stable and smooth.

It is running in automatic mode, all necessary devices in my network are activated.

So i can now exclude the possibility that my network is a reason.

but if you need further testing, i will try to help with the old pi3b+

@safri Raspi 4 has no issues at all. Some Raspi 3 run fine without any problems too. Seems that the issue is happening on certain hardware releases only.

We are clueless at the moment as the image is the same for all Raspi 🤔 If you feel like helping, please send your „old“ Raspi 3 to us (see imprint).

Any ideas are highly appreciated.

THX very much!

I have exactly the same problems with an Rasp 2 Model B V.1. 1 GB RAM on eblocker 2.5.8. Power supply is 3V

Before 2.5 everything worked fine. When I start the eblocker it works okay for a few hours and then doesn't respond anymore.

Only devices in the LAN which use the https certificate seem to be affected, I have an iPhone that has the https certificate installed and it's not affected by this problem. 

Is this a known problem?

At least I have found a way to reproduce the problem within a few minutes. A rooted eBlocker and a computer with netcat or nc is needed.

For example, on macOS:

Log in on (rooted) eBlocker and start a listening process on port 1234:

nc -l -p 1234 > /dev/null

Start "Activity Monitor" on macOS and go to tab "Network". Observe the packet graph.

Open "Terminal" and send null bytes to the eBlocker as fast as possible:

nc eblocker.box 1234 < /dev/zero

The packet graph should show large amounts of sent data per second (the red curve). After about a minute, it stops. The Raspberry Pi is still running, but it has lost all network connection. It cannot be pinged.

The kernel has written a warning to /var/log/syslog:

NETDEV WATCHDOG: eth0 (lan78xx): transmit queue 0 timed out

Next, I will test a plain Raspbian system without eBlocker software.

I was finally able to reproduce the issue on a plain Raspbian system without any eBlocker software.

Here are the steps:

• Install Raspbian on Raspberry Pi 3B+
• Upgrade to latest version: apt-get update && apt-get dist-upgrade
• Reboot
• Enable IP forwarding: sudo sysctl net.ipv4.ip_forward=1
• Enable masquerading: sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
• Start listening process: sudo nc -l -p 1234 > /dev/null

On another computer (Linux or macOS):

• Configure IPv4 address manually setting Raspberry Pi as gateway
• Open browser and load some web pages to test routing
• Start load test: nc EBLOCKER_IP 1234 < /dev/zero

Parallel browsing now leads to network crash on Raspberry Pi. The kernel warns:

transmit queue 0 timed out

@hellb @safri @lbausk @serpentin @pio78

To sum up the current findings:

• The freeze issue is related to a bug in the current Rasberry Pi OS Lite (formerly Raspian) with kernel 5.4.79 (which eBlockerOS 2.5.x uses as base OS)
• The bug occurs if certain firewall rules in the RaspiOS are set (see post from @bpr above). These rules are eBlocker standard and can not be omitted to run eOS.
• Unfortunately the bug can not be fixed by eBlocker but needs to be revised by raspberrypi.org (we've filed a bug report)
• The bug only affects Raspberry Pi 3B+ (Rasberry Pi 2 and Raspberry Pi 3 affected)

@valentin As your Raspberry Pi 3 runs eBlockerOS 2.5.x without a problem, can you please share the exact model and RAM. Seems there is a chipset change for the 3B+, that causes the problem.

At the very moment we recommend to use Rasberry Pi 2 or Raspberry Pi 4 for running eBlockerOS (and not use it on Raspi 3B+).

We'll update this thread as soon as we hear back from raspberrypi.org.

Sorry for the issues but it's out of our control.

@benne Thanks for your investigation. As I wrote above, I use a Raspi 2 (Model B V.1. 1 GB RAM) and have the same problem, how ist that possible?

@benne well, my Raspi 3+ was running eBlocker without big problems in version 2.5.6. But I also have the experience that since 2.5.8 from time to time the eBlocker stops with exactly the same error picture: no WEB, no SSH, no ping anymore. I can only do a cold boot.

Some days ago, I had to reactive it 3 times within 24h. But I always have been thinking that it is the known memory problem so far. My time to investigate currently is limited due to private and job reasons, but  things will become better the next 3 weeks...

My Raspi 3B+ has 1 GB memory. Even if eBlocker does not track anything (devices switched off), it consumes at least 790MB of it. There is really not much memory left to be active.

Posted by: @hellb

have the same problem

I‘m not sure whether it‘s the same reason, even if it appears that you have the „same“ problem.

Posted by: @hellb

Before 2.5 everything worked fine. When I start the eblocker it works okay for a few hours and then doesn't respond anymore.

Only devices in the LAN which use the https certificate seem to be affected,

To me this sounds like a different cause then. Anyway, I‘d try to remove extra external DNS you might have added (or switch off DNS firewall for the test).

Can you please check your network connection to verify routing next time you can not connect to the Internet: https://eblocker.org/docs/check-the-network-connection-with-windows/ (same doc available for macos).

Please post results here. 

THX!

@hellb

Do you have a Raspberry Pi 2 v1.1 or v1.2?

The chip was changed from a BCM2836 to BCM2837 (according to https://en.wikipedia.org/wiki/Raspberry_Pi ).

On my Pi 2 v1.1 with BCM2836 the problem did not occur. I ran a load test for over half an hour. (On the 3B+ the error occurred always within 90 seconds.)

Posted by: @bpr

@hellb

Do you have a Raspberry Pi 2 v1.1 or v1.2?

The chip was changed from a BCM2836 to BCM2837 (according to https://en.wikipedia.org/wiki/Raspberry_Pi ).

On my Pi 2 v1.1 with BCM2836 the problem did not occur. I ran a load test for over half an hour. (On the 3B+ the error occurred always within 90 seconds.)

I have an Rasp 2 Model B V.1. 1. The problem happens after a few hours, I'll try the tips that Benne and Random told me. Perhaps it's an issue with the Cookie banner block that I added some time ago, i'll gibe it a try.

Hi everybody,

thanks to @mainzelm and @bpr I now have my Raspi 3B+ running with an external USB to LAN adapter (its another chip). It seems to be working again since some days. But from time to time, ICAP still had been stopping. So I reduced the memory print by changing the java call to

java -Xmx256m -Dlog4j.configuration=file:///opt/eblocker-icap/conf/icapserver-log4j.properties

Now I have an average total memory print of 740M out of 925M and this seems to be stable.

From the crashes, I collected some messages out of eBlocker log but they are too big to be posted here. There is an evidence for (maybe garbage driven) memory loss under several circumstances which possibly could affect other Raspi's as well. Maybe this is an offline topic for email.

Again, I found that in version 2.5.6 ICAP was working fine but in 2.5.8 it did not. I think the image footprint grow up and a critical 1GB memory condition is violated in the default 2.5.8 version

@valentin

I‘m currently following a hot track that might explain the memory problems you‘re experiencing, so stay tuned...

Best regards

Martin

@valentin

I think, I found good explanation for your problem. To verify my assumption, could you please revert your -Xmx setting, and add the following setting:

-XX:MaxDirectMemorySize=100m

and then restart the Icapserver? I'd guess that your eBlocker runs stable again with this flag added. Please tell me about your experiences with it.

Best regards,

Martin

@all Meanwhile the linux bug we've reported has changed status to "confirmed" by Raspian. So chances are good we can support Raspberry Pi 3 in the near future again. But unfortunately we have no influence how quick they fix the kernel - and just hope for the best.

As soon as we got news, we'll share them here.

@all

eOS 2.6 comes with the latest linux kernel which apparently fixes the „freeze bug“. 😀 😎 

So eBlocker 2.6 will work on Raspi 3 again. Very good job @bpr!

As soon as the update has been released we will provide a new SD card image as well.

Update is due in a few days... 🚀

eBlockerOS 2.6 has been released: Download 

THX for your patience! 😎