Merge branch '8.8RC'

work with Mastodon-style keyId

See merge request hubzilla/core!2060

.debianinstall/README.md

@@ -0,0 +1,158 @@
+
+# How to use
+
+## Disclaimers
+
+- **This script does work with a fresh install of Debian 12 only**.
+- Do not use if you have already installed and configured a webserver or sql server that was not installed by this script.
+
+### Keep it Simple and Stupid
+
+The script keeps everything as simple as possible (KISS):
+
+- Apache as webserver (there is no choice to use another webserver like nginx)
+- default PHP version of Debian
+- one single Hubzilla intance only
+- re-running the script does no harm
+
+### When to use other Scripts
+
+Use the scripts under [homeinstall](https://framagit.org/hubzilla/core/-/tree/master/.homeinstall)
+if you look for more choices. The main differences are:
+
+- Apache or nginx as webserver
+- install multiple instances (domains) that run side by side on the server
+- adds apache vhosts (instead of using the standard doc root /var/www/html)
+- install PHP from https://packages.sury.org/php/ (instead of using the Debian repository)
+- graphical installer whiptail
+- The script stops (fails) if it finds results of a previous installation. (The [debian-setup.sh](https://framagit.org/ojrandom/core/-/tree/dev/.debianinstall) will just jump over it.)
+- If something fails the script tries to clean up everything that was installed up to the point of failure. (That might cause trouble if certbot registered a certificate already.)
+- The script under [homeinstall](https://framagit.org/hubzilla/core/-/tree/master/.homeinstall) seems to be an older version of the scripts used for Streams
+  + [autoinstall](https://codeberg.org/streams/streams/src/branch/dev/contrib/autoinstall)
+  + [easyinstall](https://codeberg.org/streams/streams/src/branch/dev/contrib/easyinstall)
+
+## Preconditions
+
+Hardware
+
++ internet connection and router at home
++ computer connected to your router (a Raspberry 3 will do for very small Hubs)
+
+Software
+
++ fresh installation of Debian 12 (bookworm)
++ router with open ports 80 and 443 for your web server
+
+You can of course run the script on a VPS or any distant server as long as the above sotfware requirements are satisfied.
+
+## How to run the script
+
++ Register your own domain (for example at selfHOST) or a free subdomain (for example at freeDNS)
++ Log on to your fresh Debian
+  - apt-get install git
+  - mkdir -p /var/www
+  - cd /var/www
+  - git clone https://framagit.org/hubzilla/core.git html
+  - cd html/.debianinstall
+  - cp config.txt.template config.txt
+  - nano config.txt
+    - read the comments carefully
+    - enter your values: db pass, domain
+    - (optionally) Enter your values for dyn DNS
+  - ./debian-setup.sh as root
+    - ... wait, wait, wait until the script is finished
++ Open your domain with a browser and step throught the initial configuration of your hubzilla instance.
+  - default database name = hubzilla
+  - default dababase user = hubzilla
+
+## Optional - Switch verification of email on/off
+
+Do this just before you register the first user without email verification.
+
+In a terminal
+
+    su -
+    cd /var/www/html
+
+Check the current setting 
+
+    util/config system verify_email
+
+Switch the verification off
+
+    util/config system verify_email 0
+
+## What the script will do for you...
+
++ install everything required by your hubzilla instance, basically a web server (Apache), PHP, a database (MySQL), certbot,...
++ create a database
++ run certbot to have everything for a secure connection (httpS)
++ create a script for daily maintenance
+  - renew certfificate (letsencrypt)
+  - update of your hubzilla instance for core and addons (git)
+  - update of Debian
+  - restart
++ create cron jobs for
+  - DynDNS (selfHOST.de or freedns.afraid.org) every 5 minutes
+  - Master.php for your hubzilla instance every 10 minutes
+  - daily maintenance script every day at 05:30
+
+The script is known to work without adjustments with
+
++ Hardware
+  - standard PC with Debian 12 (bookworm)
+  - Raspberry 4 with Raspbian, Debian 12 (TODO: needs confirmation after swich to Debian12)
+  - for tesing purposes: under localhost inside a virtual machine, [KVM](https://wiki.debian.org/KVM)
++ DynDNS
+  - selfHOST.de
+  - freedns.afraid.org
+
+# Step-by-Step - some Details
+
+## Preparations
+
+## Configure your Router
+
+Your webserver has to be visible in the internet.  
+
+Open the ports 80 and 443 on your router for your Debian. Make sure your web server is marked as "exposed host".
+
+## Preparations Dynamic IP Address
+
+Follow the instructions in .debianinstall/config.txt.  
+
+In short...  
+
+Your Hubzilla server must be reachable by a domain that you can type in your browser
+
+    cooldomain.org
+
+You can use subdomains as well
+
+    my.cooldomain.org
+
+There are two ways to get a domain...
+
+### Method 1: Buy a Domain 
+
+...for example buy at selfHOST.de  
+
+The cost is 1,50 € per month (2019).
+
+### Method 2: Register a free subdomain
+
+...for example register at freedns.afraid.org
+
+## Note on Rasperry 
+
+It is recommended to run the Raspi without graphical frontend (X-Server). Use...
+
+    sudo raspi-config
+
+to boot the Rapsi to the client console.
+
+DO NOT FORGET TO CHANGE THE DEFAULT PASSWORD FOR USER PI!
+
+## Reminder for Different Web Wervers
+
+For those of you who feel adventurous enough to use a different web server (i.e. Lighttpd...), don't forget that this script will install Apache or Nginx and that you can only have one web server listening to ports 80 & 443. Also, don't forget to tweak your daily shell script in /var/www/ accordingly.

.debianinstall/config.txt.template

@@ -0,0 +1,111 @@
+
+###############################################
+### MANDATORY - database password #############
+#
+# Please give your database password
+# It is better to not use blanks inside the password.
+#   Example: db_pass=pass_word_with_no_blanks_in_it
+db_pass=
+
+###############################################
+### MANDATORY - let's encrypt #################
+#
+# Zot requires encrypted communication via secure HTTP (HTTPS).
+# This script automates installation of an SSL certificate from
+# Let's Encrypt (https://letsencrypt.org)
+#
+# Please give the domain name of your hub/instance
+#
+#     Example: my.cooldomain.org
+#     Example: cooldomain.org
+#
+# You might use "localhost" for a LOCAL TEST installation.
+# This is usefull if you want to debug the server inside a VM.
+#
+#     Example: localhost
+#
+# Email is optional if you use "localhost".
+#
+#
+le_domain=
+le_email=
+
+
+###############################################
+### OPTIONAL - selfHOST - dynamic IP address ##
+#
+# 1. Register a domain at selfhost.de
+#    - choose offer "DOMAIN dynamisch" 1,50€/mon at 04/2019
+# 2. Get your configuration for dynamic IP update
+#    - Log in at selfhost.de
+#    - go to "DynDNS Accounte"
+#    - klick "Details" of your (freshly) registered domain
+#    - You will find the configuration there
+#      - Benutzername (user name) > use this for "selfhost_user="
+#      - Passwort (pass word) > use this for "selfhost_pass="
+#
+#
+selfhost_user=
+selfhost_pass=
+
+###############################################
+### OPTIONAL - FreeDNS - dynamic IP address ###
+#
+# Please give the alpha-numeric-key of freedns
+#
+# Get a free subdomain from freedns and use it for your dynamic ip address
+# Documentation under http://www.techjawab.com/2013/06/setup-dynamic-dns-dyndns-for-free-on.html
+#
+# - Register for a Free domain at http://freedns.afraid.org/signup/
+# - WATCH THIS: Make sure you choose a domain with as less subdomains as
+#   possible. Why? Let's encrpyt issues a limited count of certificates each
+#   day. Possible other users of this domain will try to issue a certificate
+#   at the same day.
+# - Logon to FreeDNS (where you just registered)
+# - Goto http://freedns.afraid.org/dynamic/
+# - Right click on "Direct Link" and copy the URL and paste it somewhere.
+# - You should notice a large and unique alpha-numeric key in the URL
+#
+#       http://freedns.afraid.org/dynamic/update.php?alpha-numeric-key
+#
+#   Provided your url from freedns is
+#
+#	http://freedns.afraid.org/dynamic/update.php?U1Z6aGt2R0NzMFNPNWRjbWxxZGpsd093OjE1Mzg5NDE5
+#
+#   Then you have to provide
+#
+#       freedns_key=U1Z6aGt2R0NzMFNPNWRjbWxxZGpsd093OjE1Mzg5NDE5
+#
+#
+freedns_key=
+
+
+###############################################
+### OPTIONAL - do not mess with things below ##
+#              (...if you are not certain)
+#
+# Usually you are done here
+# Everything below is OPTIONAL
+#
+###############################################
+#
+# Database for your hub/instance
+# If left empty, both your database and user will be named after your zot instance (hubzilla, zap or misty)
+# Use custom name, at least fo the database, if you plan to run more than one hub/instance on the same server
+#
+zotserver_db_name=
+zotserver_db_user=
+zotserver_db_pass=$db_pass
+# 
+#
+# Password for package mysql-server
+#   Example: mysqlpass=aberhallo
+#   Example: mysqlpass="aber hallo has blanks in it"
+#
+mysqlpass=$db_pass
+
+# Password for package phpmyadmin
+#   Example: phpmyadminpass=aberhallo
+#   Example: phpmyadminpass="aber hallo has blanks in it"
+phpmyadminpass=$db_pass
+

.debianinstall/debian-setup.sh

@@ -0,0 +1,527 @@
+#!/bin/bash
+#
+# How to use
+# ----------
+#
+# This file automates the installation of hubzilla: https://framagit.org/hubzilla/core
+# under Debian Linux "bookworm"
+#
+# 1) Copy the file "config.txt.template" to "config.txt"
+#       Follow the instuctions there
+#
+# 2) Switch to user "root" by typing "su -"
+#
+# 3) Run with "./debian-setup.sh"
+#       If this fails check if you can execute the script.
+#       - To make it executable type "chmod +x debian-setup.sh"
+#       - or run "bash debian-setup.sh"
+#
+#
+# What does this script do basically?
+# -----------------------------------
+#
+# This file automates the installation of a Hubzilla instance under Debian Linux
+# - install
+#        * apache webserver,
+#        * php,
+#        * mariadb,
+#        * adminer,
+#        * addons
+# - configure cron
+#        * "Master.php" for regular background processes of your hubzilla instance
+#        * "apt-get update" and "apt-get dist-upgrade" and "apt-get autoremove" to keep linux up-to-date
+#        * run command to keep the IP up-to-date > DynDNS provided by selfHOST.de or freedns.afraid.org
+# - run letsencrypt to create, register and use a certifacte for https
+#
+#
+# Credits
+# -------
+#
+# The script is derived from the easyinstall script of the Streams repository, which is based on
+# - Tom Wiedenhöfts (OJ Random) script homeinstall (for Hubzilla, ZAP,...) that was based on
+# - Thomas Willinghams script "debian-setup.sh" which he used to install the red#matrix.
+
+function check_sanity {
+    # Do some sanity checking.
+    print_info "Sanity check..."
+    if [ $(/usr/bin/id -u) != "0" ]
+    then
+        die 'Must be run by root user'
+    fi
+
+    if [ -f /etc/lsb-release ]
+    then
+        die "Distribution is not supported"
+    fi
+    if [ ! -f /etc/debian_version ]
+    then
+        die "Debian is supported only"
+    fi
+    if ! grep -q 'Linux 12' /etc/issue
+    then
+        die "Linux 12 (bookworm) is supported only"x
+    fi
+}
+
+function check_config {
+    print_info "config check..."
+    # Check for required parameters
+    if [ -z "$db_pass" ]
+    then
+        die "db_pass not set in $configfile"
+    fi
+    if [ -z "$le_domain" ]
+    then
+        die "le_domain not set in $configfile"
+    fi
+}
+
+function die {
+    echo "ERROR: $1" > /dev/null 1>&2
+    exit 1
+}
+
+
+function update_upgrade {
+    print_info "updated and upgrade..."
+    # Run through the apt-get update/upgrade first. This should be done before
+    # we try to install any package
+    apt-get -q -y update && apt-get -q -y dist-upgrade && apt-get -q -y autoremove
+    print_info "updated and upgraded linux"
+}
+
+function nocheck_install {
+    # export DEBIAN_FRONTEND=noninteractive ... answers from the package configuration database
+    # - q ... without progress information
+    # - y ... answer interactive questions with "yes"
+    # DEBIAN_FRONTEND=noninteractive apt-get --no-install-recommends -q -y install $2
+    # DEBIAN_FRONTEND=noninteractive apt-get --install-suggests -q -y install $1
+    DEBIAN_FRONTEND=noninteractive apt-get -q -y install $1
+    print_info "installed $1"
+}
+
+
+function print_info {
+    echo -n -e '\e[1;34m'
+    echo -n $1
+    echo -e '\e[0m'
+}
+
+function print_warn {
+    echo -n -e '\e[1;31m'
+    echo -n $1
+    echo -e '\e[0m'
+}
+
+function stop_zotserver {
+    print_info "stopping apache..."
+    systemctl stop apache2
+    print_info "stopping mysql db..."
+    systemctl stop mariadb
+}
+
+function install_apache {
+    print_info "installing apache..."
+    nocheck_install "apache2 apache2-utils"
+    a2enmod rewrite
+    systemctl restart apache2
+}
+
+function install_imagemagick {
+    print_info "installing imagemagick..."
+    nocheck_install "imagemagick"
+}
+
+function install_curl {
+    print_info "installing curl..."
+    nocheck_install "curl"
+}
+
+function install_wget {
+    print_info "installing wget..."
+    nocheck_install "wget"
+}
+
+function install_sendmail {
+    print_info "installing sendmail..."
+    nocheck_install "sendmail sendmail-bin"
+}
+
+function install_php {
+    # openssl and mbstring are included in libapache2-mod-php
+    print_info "installing php..."
+    nocheck_install "libapache2-mod-php php php-pear php-curl php-gd php-mbstring php-xml php-zip"
+    phpversion=$(php -v|grep --only-matching --perl-regexp "(PHP )\d+\.\\d+\.\\d+"|cut -c 5-7)
+    sed -i "s/^upload_max_filesize =.*/upload_max_filesize = 100M/g" /etc/php/$phpversion/apache2/php.ini
+    sed -i "s/^post_max_size =.*/post_max_size = 100M/g" /etc/php/$phpversion/apache2/php.ini
+}
+
+function install_composer {
+    print_info "We check if Composer is already downloaded"
+    if [ ! -f /usr/local/bin/composer ]
+    then
+        EXPECTED_CHECKSUM="$(php -r 'copy("https://composer.github.io/installer.sig", "php://stdout");')"
+        php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
+        ACTUAL_CHECKSUM="$(php -r "echo hash_file('sha384', 'composer-setup.php');")"
+        if [ "$EXPECTED_CHECKSUM" != "$ACTUAL_CHECKSUM" ]
+        then
+            >&2 echo 'ERROR: Invalid installer checksum'
+            rm composer-setup.php
+            die 'ERROR: Invalid installer checksum'
+        fi
+        php composer-setup.php --quiet
+        RESULT=$?
+        rm composer-setup.php
+        # exit $RESULT
+        # We install Composer globally
+        mv composer.phar /usr/local/bin/composer
+        print_info "Composer was successfully downloaded."
+    else
+        print_info "Composer is already downloaded on this system."
+    fi
+    cd $install_path
+    export COMPOSER_ALLOW_SUPERUSER=1;
+    /usr/local/bin/composer install --no-dev
+    /usr/local/bin/composer show
+    export COMPOSER_ALLOW_SUPERUSER=0;
+}
+
+
+function install_mysql {
+    print_info "installing mysql..."
+    if [ -z "$mysqlpass" ]
+    then
+        die "mysqlpass not set in $configfile"
+    fi
+	if type mysql ; then
+        echo "Yes, mysql is installed"
+	else
+        echo "mariadb-server"
+        nocheck_install "mariadb-server"
+        systemctl status mariadb
+        systemctl start mariadb
+        mysql --user=root <<_EOF_
+UPDATE mysql.user SET Password=PASSWORD('${mysqlpass}') WHERE User='root';
+DELETE FROM mysql.user WHERE User='';
+DROP DATABASE IF EXISTS test;
+DELETE FROM mysql.db WHERE Db='test' OR Db='test\\_%';
+FLUSH PRIVILEGES;
+_EOF_
+    fi
+}
+
+function install_adminer {
+    print_info "installing adminer..."
+    nocheck_install "adminer"
+    if [ ! -f /etc/adminer/adminer.conf ]
+    then
+        echo "Alias /adminer /usr/share/adminer/adminer" > /etc/adminer/adminer.conf
+        ln -s /etc/adminer/adminer.conf /etc/apache2/conf-available/adminer.conf
+    else
+        print_info "file /etc/adminer/adminer.conf exists already"
+    fi
+
+    a2enmod rewrite
+
+    if [ ! -f /etc/apache2/apache2.conf ]
+    then
+        die "could not find file /etc/apache2/apache2.conf"
+    fi
+    sed -i \
+        "s/AllowOverride None/AllowOverride all/" \
+        /etc/apache2/apache2.conf
+
+    a2enconf adminer
+    systemctl restart mariadb
+    systemctl reload apache2
+}
+
+function create_zotserver_db {
+    print_info "creating zotserver database..."
+    if [ -z "$db_name" ]
+    then
+        die "db_name not set in $configfile"
+    fi
+    if [ -z "$db_user" ]
+    then
+        die "db_user not set in $configfile"
+    fi
+    if [ -z "$db_pass" ]
+    then
+        die "db_pass not set in $configfile"
+    fi
+    systemctl restart mariadb
+    # Make sure we don't write over an already existing database if we install more than one Zot hub/instance
+    if [ -z $(mysql -h localhost -u root -p$mysqlpass -e "SHOW DATABASES;" | grep $db_name) ]
+    then
+        Q1="CREATE DATABASE IF NOT EXISTS $db_name;"
+        Q2="GRANT USAGE ON *.* TO $db_user@localhost IDENTIFIED BY '$db_pass';"
+        Q3="GRANT ALL PRIVILEGES ON $name.* to $db_user@localhost identified by '$db_pass';"
+        Q4="FLUSH PRIVILEGES;"
+        SQL="${Q1}${Q2}${Q3}${Q4}"
+        mysql -uroot -p$mysqlpass -e "$SQL"
+    else
+        echo "database $db_name does exist already"
+    fi
+}
+
+function run_freedns {
+    print_info "run freedns (dynamic IP)..."
+    if [ -z "$freedns_key" ]
+    then
+        print_info "freedns was not started because 'freedns_key' is empty in $configfile"
+    else
+        if [ -n "$selfhost_user" ]
+        then
+            die "You can not use freeDNS AND selfHOST for dynamic IP updates ('freedns_key' AND 'selfhost_user' set in $configfile)"
+        fi
+        wget --no-check-certificate -O - http://freedns.afraid.org/dynamic/update.php?$freedns_key
+    fi
+}
+
+function install_run_selfhost {
+    print_info "install and start selfhost (dynamic IP)..."
+    if [ -z "$selfhost_user" ]
+    then
+        print_info "selfHOST was not started because 'selfhost_user' is empty in $configfile"
+    else
+        if [ -n "$freedns_key" ]
+        then
+            die "You can not use freeDNS AND selfHOST for dynamic IP updates ('freedns_key' AND 'selfhost_user' set in $configfile)"
+        fi
+        if [ -z "$selfhost_pass" ]
+        then
+            die "selfHOST was not started because 'selfhost_pass' is empty in $configfile"
+        fi
+        if [ ! -d $selfhostdir ]
+        then
+            mkdir $selfhostdir
+        fi
+        # the old way
+        # https://carol.selfhost.de/update?username=123456&password=supersafe
+        #
+        # the prefered way
+        wget --output-document=$selfhostdir/$selfhostscript http://jonaspasche.de/selfhost-updater
+        echo "router" > $selfhostdir/device
+        echo "$selfhost_user" > $selfhostdir/user
+        echo "$selfhost_pass" > $selfhostdir/pass
+        bash $selfhostdir/$selfhostscript update
+    fi
+}
+
+function ping_domain {
+    print_info "ping domain $domain..."
+    # Is the domain resolved? Try to ping 6 times à 10 seconds
+    COUNTER=0
+    for i in {1..6}
+    do
+        print_info "loop $i for ping -c 1 $domain ..."
+        if ping -c 4 -W 1 $le_domain
+        then
+            print_info "$le_domain resolved"
+            break
+        else
+            if [ $i -gt 5 ]
+            then
+                die "Failed to: ping -c 1 $domain not resolved"
+            fi
+        fi
+        sleep 10
+    done
+    sleep 5
+}
+
+function configure_cron_freedns {
+    print_info "configure cron for freedns..."
+    if [ -z "$freedns_key" ]
+    then
+        print_info "freedns is not configured because freedns_key is empty in $configfile"
+    else
+        # Use cron for dynamich ip update
+        #   - at reboot
+        #   - every 30 minutes
+        if [ -z "`grep 'freedns.afraid.org' /etc/crontab`" ]
+        then
+            echo "@reboot root http://freedns.afraid.org/dynamic/update.php?$freedns_key > /dev/null 2>&1" >> /etc/crontab
+            echo "*/30 * * * * root wget --no-check-certificate -O - http://freedns.afraid.org/dynamic/update.php?$freedns_key > /dev/null 2>&1" >> /etc/crontab
+        else
+            print_info "cron for freedns was configured already"
+        fi
+    fi
+}
+
+function configure_cron_selfhost {
+    print_info "configure cron for selfhost..."
+    if [ -z "$selfhost_user" ]
+    then
+        print_info "selfhost is not configured because selfhost_key is empty in $configfile"
+    else
+        # Use cron for dynamich ip update
+        #   - at reboot
+        #   - every 5 minutes
+        if [ -z "`grep 'selfhost-updater.sh' /etc/crontab`" ]
+        then
+            echo "@reboot root bash /etc/selfhost/selfhost-updater.sh update > /dev/null 2>&1" >> /etc/crontab
+            echo "*/5 * * * * root /bin/bash /etc/selfhost/selfhost-updater.sh update > /dev/null 2>&1" >> /etc/crontab
+        else
+            print_info "cron for selfhost was configured already"
+        fi
+    fi
+}
+
+function install_letsencrypt {
+    print_info "installing let's encrypt ..."
+    # check if user gave domain
+    if [ -z "$le_domain" ]
+    then
+        die "Failed to install let's encrypt: 'le_domain' is empty in $configfile"
+    fi
+    if [ -z "$le_email" ]
+    then
+        die "Failed to install let's encrypt: 'le_email' is empty in $configfile"
+    fi
+    nocheck_install "certbot python-certbot-apache"
+    print_info "run certbot ..."
+    certbot --apache -w $install_path -d $le_domain -m $le_email --agree-tos --non-interactive --redirect --hsts --uir
+    service apache2 restart
+}
+
+function check_https {
+    print_info "checking httpS > testing ..."
+    url_https=https://$le_domain
+    wget_output=$(wget -nv --spider --max-redirect 0 $url_https)
+    if [ $? -ne 0 ]
+    then
+        print_warn "check not ok"
+    else
+        print_info "check ok"
+    fi
+}
+
+function install_zotserver {
+    print_info "installing addons..."
+    cd $install_path
+    util/add_addon_repo https://framagit.org/hubzilla/addons hzaddons
+    mkdir -p "store/[data]/smarty3"
+    # chmod -R 777 store
+    touch .htconfig.php
+    # The next run of $cron_job (daily-update script) will correct the permissions of the next line
+    chmod ou+w .htconfig.php  
+    cd /var/www/
+    chown -R www-data:www-data $install_path
+	chown root:www-data $install_path/
+	chown root:www-data $install_path/.htaccess
+	chmod 0644 $install_path/.htaccess
+    print_info "installed addons"
+}
+
+function configure_cron_daily {
+    print_info "configuring cron..."
+    # every 10 min for poller.php
+    if [ -z "`grep 'php Zotlabs/Daemon/Master.php' /etc/crontab`" ]
+    then
+        echo "*/10 * * * * www-data cd $install_path; php Zotlabs/Daemon/Master.php Cron >> /dev/null 2>&1" >> /etc/crontab
+    fi
+    # Run external script daily at 05:30
+    # - stop apache/nginx and mysql-server
+    # - renew the certificate of letsencrypt
+    # - update repository core and addon
+    # - update and upgrade linux
+    # - reboot is done by "shutdown -h now" because "reboot" hangs sometimes depending on the system
+    echo "#!/bin/sh" > /var/www/$cron_job
+    echo "#" >> /var/www/$cron_job
+    echo "echo \" \"" >> /var/www/$cron_job
+    echo "echo \"+++ \$(date) +++\"" >> /var/www/$cron_job
+    echo "echo \" \"" >> /var/www/$cron_job
+    echo "echo \"\$(date) - stopping apache and mysql...\"" >> /var/www/$cron_job
+    echo "service apache2 stop" >> /var/www/$cron_job
+    echo "/etc/init.d/mysql stop # to avoid inconsistencies" >> /var/www/$cron_job
+    echo "#" >> /var/www/$cron_job
+    echo "echo \"\$(date) - renew certificate...\"" >> /var/www/$cron_job
+    echo "certbot renew --noninteractive" >> /var/www/$cron_job
+    echo "#" >> /var/www/$cron_job
+    echo "echo \"\$(date) - db size...\"" >> /var/www/$cron_job
+    echo "du -h /var/lib/mysql/ | grep mysql/" >> /var/www/$cron_job
+    echo "#" >> /var/www/$cron_job    
+    echo "# update of $le_domain Zot hub/instance" >> /var/www/$cron_job
+    echo "echo \"\$(date) - updating core and addons...\"" >> /var/www/$cron_job
+    echo "echo \"reaching git repository for $le_domain $zotserver hub/instance...\"" >> /var/www/$cron_job
+    echo "(cd $install_path ; util/udall)" >> /var/www/$cron_job
+    echo "chown -R www-data:www-data $install_path # make all accessible for the webserver" >> /var/www/$cron_job
+    echo "chown root:www-data $install_path/.htaccess" >> /var/www/$cron_job
+    echo "chmod 0644 $install_path/.htaccess # www-data can read but not write it" >> /var/www/$cron_job    
+    echo "echo \"\$(date) - updating linux...\"" >> /var/www/$cron_job
+    echo "apt-get -q -y update && apt-get -q -y dist-upgrade && apt-get -q -y autoremove # update linux and upgrade" >> /var/www/$cron_job
+    echo "echo \"\$(date) - Update finished. Rebooting...\"" >> /var/www/$cron_job
+    echo "#" >> /var/www/$cron_job
+    echo "shutdown -r now" >> /var/www/$cron_job
+
+    chmod a+x /var/www/$cron_job
+
+    # If global cron job does not exist we add it to /etc/crontab
+    if grep -q $cron_job /etc/crontab
+    then
+        echo "cron job already in /etc/crontab"
+    else
+        echo "30 05 * * * root /bin/bash /var/www/$cron_job >> /var/www/daily-updates.log 2>&1" >> /etc/crontab
+        echo "0 0 1 * * root rm /var/www/daily-updates.log" >> /etc/crontab
+    fi
+
+    # This is active after either "reboot" or cron reload"
+    systemctl restart cron
+    print_info "configured cron for updates/upgrades"
+}
+
+########################################################################
+# START OF PROGRAM
+########################################################################
+export PATH=/bin:/usr/bin:/sbin:/usr/sbin
+check_sanity
+
+print_info "We're installing a $zotserver instance"
+install_path="$(dirname "$(pwd)")"
+
+# Read config file edited by user
+configfile=config.txt
+source $configfile
+
+selfhostdir=/etc/selfhost
+selfhostscript=selfhost-updater.sh
+cron_job="cron_job.sh"
+
+#set -x    # activate debugging from here
+
+zotserver=hubzilla
+check_config
+stop_zotserver
+update_upgrade
+install_curl
+install_wget
+install_sendmail
+install_apache
+install_imagemagick
+install_php
+install_composer
+install_mysql
+install_adminer
+create_zotserver_db
+run_freedns
+install_run_selfhost
+ping_domain
+configure_cron_freedns
+configure_cron_selfhost
+
+if [ "$le_domain" != "localhost" ]
+then
+    install_letsencrypt
+    check_https
+else
+    print_info "is localhost - skipped installation of letsencrypt and configuration of apache for https"
+fi
+
+install_zotserver
+
+configure_cron_daily
+
+
+#set +x    # stop debugging from here

.gitlab-ci.yml

@@ -1,7 +1,9 @@
 # Select image from https://hub.docker.com/_/php/
 #image: php:7.3
 # Use a prepared Hubzilla image to optimise pipeline duration
-image: registry.gitlab.com/dawnbreak/hubzilla/core:php7.3
+# image: registry.gitlab.com/dawnbreak/hubzilla/core:php7.3
+
+image: php:8.1
 
 stages:
   - test
@@ -35,7 +37,11 @@ before_script:
 - if [ -f /usr/local/etc/php/conf.d/z_prod.ini ]; then mv /usr/local/etc/php/conf.d/z_prod.ini /usr/local/etc/php/conf.d/z_prod.ini.off; fi
 # Install & enable Xdebug for code coverage reports
 - pecl install xdebug
+- apt-get update
+- apt-get install zip unzip libjpeg-dev libpng-dev -yqq
 - docker-php-ext-enable xdebug
+- docker-php-ext-install gd
+
 # Install composer
 - curl -sS https://getcomposer.org/installer | php
 # Install dev libraries from composer
@@ -43,31 +49,38 @@ before_script:
 # php.ini settings
 - echo 'xdebug.mode=coverage' >> /usr/local/etc/php/php.ini
 
-# hidden job definition with template for MySQL/MariaDB
-.job_template_mysql: &job_definition_mysql
+# hidden job definition with template for PHP
+.job_template_php: &job_definition_php
   stage: test
   script:
-  - echo "USE $MYSQL_DATABASE; $(cat ./install/schema_mysql.sql)" | mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mysql "$MYSQL_DATABASE"
-  - echo "SHOW DATABASES;" | mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mysql "$MYSQL_DATABASE"
-  - echo "USE $MYSQL_DATABASE; SHOW TABLES;" | mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mysql "$MYSQL_DATABASE"
   - vendor/bin/phpunit --configuration tests/phpunit.xml --coverage-text
 
+
+# hidden job definition with template for MySQL/MariaDB
+#.job_template_mysql: &job_definition_mysql
+#  stage: test
+#  script:
+#  - echo "USE $MYSQL_DATABASE; $(cat ./install/schema_mysql.sql)" | mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mysql "$MYSQL_DATABASE"
+#  - echo "SHOW DATABASES;" | mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mysql "$MYSQL_DATABASE"
+#  - echo "USE $MYSQL_DATABASE; SHOW TABLES;" | mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mysql "$MYSQL_DATABASE"
+#  - vendor/bin/phpunit --configuration tests/phpunit.xml --coverage-text
+
 # hidden job definition with template for PostgreSQL
-.job_template_postgres: &job_definition_postgres
-  stage: test
-  services:
-  - postgres:latest
-  script:
-  - export PGPASSWORD=$POSTGRES_PASSWORD
-  - psql --version
-  - psql -h "postgres" -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "SELECT VERSION();"
+#.job_template_postgres: &job_definition_postgres
+#  stage: test
+#  services:
+#  - postgres:latest
+#  script:
+#  - export PGPASSWORD=$POSTGRES_PASSWORD
+#  - psql --version
+#  - psql -h "postgres" -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "SELECT VERSION();"
   # Import hubzilla's DB schema
-  - psql -h "postgres" -U "$POSTGRES_USER" -v ON_ERROR_STOP=1 --quiet "$POSTGRES_DB" < ./install/schema_postgres.sql
+#  - psql -h "postgres" -U "$POSTGRES_USER" -v ON_ERROR_STOP=1 --quiet "$POSTGRES_DB" < ./install/schema_postgres.sql
   # Show databases and relations/tables of hubzilla's database
   #- psql -h "postgres" -U "$POSTGRES_USER" -l
   #- psql -h "postgres" -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "\dt;"
   # Run the actual tests
-  - vendor/bin/phpunit --configuration tests/phpunit-pgsql.xml --testdox
+#  - vendor/bin/phpunit --configuration tests/phpunit-pgsql.xml --testdox
 
 # hidden job definition with artifacts config template
 .artifacts_template:
@@ -82,36 +95,41 @@ before_script:
       - tests/results/
 
 
-# PHP7.3 with MySQL 5.7
-php7.3_mysql5.7:
-  <<: *job_definition_mysql
-  services:
-  - mysql:5.7
+# PHP8.1
+php8.1:
+  <<: *job_definition_php
+
+# PHP8.0 with MySQL 5.7
+#php8.0_mysql5.7:
+#  <<: *job_definition_mysql
+#  services:
+#  - mysql:5.7
 
 
-# PHP7.3 with MySQL 8 (latest)
-php7.3_mysql8:
-  <<: *job_definition_mysql
-  services:
-  - name: mysql:8
-    command: ["--default-authentication-plugin=mysql_native_password"]
+# PHP8.0 with MySQL 8 (latest)
+#php8.0_mysql8:
+#  <<: *job_definition_mysql
+#  services:
+#  - name: mysql:8
+#    command: ["--default-authentication-plugin=mysql_native_password"]
 
 
-# PHP7.3 with MariaDB 10.2
-php7.3_mariadb10.2:
-  <<: *job_definition_mysql
-  services:
-  - name: mariadb:10.2
-    alias: mysql
+# PHP8.0 with MariaDB 10.2
+#php8.0_mariadb10.2:
+#  <<: *job_definition_mysql
+#  services:
+#  - name: mariadb:10.2
+#    alias: mysql
 
 
-# PHP7.3 with MariaDB 10.3 (latest)
-php7.3_mariadb10.3:
-  <<: *job_definition_mysql
-  image: registry.gitlab.com/dawnbreak/hubzilla/core:php7.3
-  services:
-  - name: mariadb:10.3
-    alias: mysql
+# PHP8.0 with MariaDB 10.3 (latest)
+#php8.0_mariadb10.3:
+#  <<: *job_definition_mysql
+#  image: php:8.0
+  #image: registry.gitlab.com/dawnbreak/hubzilla/core:php7.3
+#  services:
+#  - name: mariadb:10.3
+#    alias: mysql
 
 
 # PHP7.3 with PostgreSQL latest (11)
@@ -131,7 +149,7 @@ php7.3_mariadb10.3:
 pages:
   stage: deploy
   cache: {}
-  image: php:7-cli-alpine
+  image: php:8-cli-alpine
   before_script:
     - apk update
     - apk add doxygen ttf-freefont graphviz

CHANGELOG

@@ -1,3 +1,524 @@
+Hubzilla 8.8 (2023-11-25)
+	- Add additional observer and channel info for nav templates
+	- Do not provide confidential channel info for templates
+	- Add link to profile to vcard
+	- Improved switch colors for better visibility
+	- Raise cache.v column from text to mediumtext for MYSQL
+	- Implement low level support for native repeats
+	- Color mode related code moved to theme_init.php in redbasic
+	- Do not overrule default list style when parsing bbcode
+	- Introduce Activity::get_actor() which will check for the cache record in xconfig before fetching
+	- Refactor Activity::actor_store()
+	- Use new language detect library which supports many more languages
+	- Use Activity::encode_person() instead of plain xchan_url to set attributedTo
+	- Update composer libraries
+	- Move right aside into bottom of left aside if screen width is < 1200px in redbasic
+	- When parsing events, use event object in first place and use the body bbcode as backup
+	- Make OWA compatible with mastodon style keyId
+	- Add sourced item events to the channel calendar
+	- Make activity filter widget portable to other modules
+	- Deprecate remains of the unused $a variable which has been replaced by the App class ages ago
+	- Return object instead of json string in the custom jsonld document loader
+	- Only show theme switch icon if switching is supported by the theme
+	- Store the actor cache date with the actor record so we can easily invalidate it after a period of time
+	- Allow themes to manipulate app icons (photos) via hook
+	- Also look for widgets in view/theme/themename/widget
+	- Cache seen pubstream item mids so that they can be hidden from notifications
+	- Theme fixes to make barebones bootstrap themes work slightly better
+	- Update Norwegian Bokmål strings
+	- Update Spanish strings
+	- Transparent background for colorbox controls icons
+	- Use body background color for colorboxes in redbasic
+
+
+	Bugfixes
+	- Remove fragment from actor urls
+	- HTTPsig case insensitive digest algorithm
+	- Fix possible privacy mismatch when processing zot requests
+	- Fix @someone in URL turned into mention in some situations - issue #1816
+	- Fix fatal error in simplepie with PHP8.2
+	- Fix tools visible allthough permission has been revoked for observer in files app
+	- Fix updates entry not removed if a channel was removed
+	- Fix form reset button not visible in mod rpost
+	- Fix missing columns for updates table in install script
+	- Fix for item widget not respecting ACL if added by title - issue #1799
+	- Fix sabre/dav caldav php warnings
+	- Fix public stream comments/reactions not allowed if item_fetched is set
+
+
+	Addons
+	- Superblock: fix php warnings
+	- Pubcrawl: restrict mod ap_probe to admin and add checkbox for signed requests
+	- Wiki: fix wiki_list widget not registered
+	- Deprecate remains of the unused $a variable which has been replaced by the App class ages ago
+	- Pubcrawl: reflect core delivery changes for repeated items
+	- Cart fix regression
+	- Logger_stats: new addon for admins to track hub activities
+	- Pubcrawl: check if we have the record in the short term cache before actually fetching it
+	- Pubcrawl: reflect core changes to Activity::actor_store()
+	- Gallery: update to photoswipe 5
+	- Pubcrawl: some platforms are sending activities without an object - return error 400
+	- HSSE: update sce-editor to latest version
+	- SSE: sse: do not process items that are older than last login date or in case we are not logged in older than 10 minutes
+	- Faces: new addon for face recognision in uploaded photos
+
+
+Hubzilla 8.6.3 (2023-09-16)
+	- Fix regression in jsonld_document_loader()
+	- Improve type checking for announce activities
+	- Improve query in drop_item() to prevent possible memory exhaustion
+	- Addon gallery: only add gallery code if the module is supported
+	- Addon hsse: port to bootstrap 5 namespaces - core issue #1793
+
+
+Hubzilla 8.6.2 (2023-08-27)
+	- Fix public stream comments/reactions fetching
+	- Fix notification text for likes in cases where obj.actor is not set
+	- Fix missing pdl file for mod cover_photo
+	- Fix unable to create folders with name 0
+	- Fix index name mixup in mysql schema file
+	- Fix missing semicolon in mysql schema file
+	- Removed unused variable
+	- Fix typo in manifest
+	- Fix cached jsonld files fetched via network
+	- Page rendering performance improvements
+	- Fix internal follow activity possibly creating notification items
+	- Fix admin table highlight color for dark mode
+	- Pubcrawl: do not handle not implemented listen activity
+	- Diaspora: fix unshare not implemented
+
+
+Hubzilla 8.6.1 (2023-07-18)
+	- Fix diaspora discovery
+
+
+Hubzilla 8.6 (2023-07-11)
+	- Update fullcalendar library
+	- Improve and unify selection of deliverable abook xchans
+	- Remove unused pseudo_abook()
+	- Implement optional moderation for unsolicited items
+	- Hardened comment permission handling for unsolicited items
+	- Remove unused templates
+	- Deprecate ActivityStreams::fetch() and provide the possibility to fetch local items directly
+	- Add simplified version of automated install script
+	- Shuffle queue deliveries for more randomness
+	- Update composer libraries
+	- Add new 2-column templates
+	- Implement optional OCAP for items to allow access to restricted media without OWA
+
+	Bugfixes
+	- Fix content in nobb and noparse text linkified - issue #1776
+	- Fix editing an event changes the set time - issue #1771
+	- Fix person object with mixed up hubloc info - issue #1770
+	- Fix $escape variable not passed on to stringify_array_elms()
+	- Fix relaying and syncing in Activity::drop()
+	- Fix allow code not sticking after channel sync - issue #1769
+	- Fix channel oembed regressions
+
+	Addons
+	- Diaspora: fix signature check for likes
+	- Diaspora: fix relaying retractions
+	- Diaspora: port to core unsolicited comments option
+	- Gallery: add an exception for streams reshares
+	- Pubcrawl: dismiss comments that are expected to arrive via owner relay
+	- Pubcrawl: improved inbox handling
+	- Pubcrawl: catch gup.pe updates to followers collection
+	- Pubcrawl: fix follow allow hook
+	- Diaspora: fix follow allow hook
+	- Content_import: fix crash in post handler
+	- Pubcrawl: store follow url when fetching webfinger
+	- Diaspora: store follow url when fetching webfinger
+	- Pubcrawl: move addressing to separate function
+
+
+Hubzilla 8.4.2 (2023-06-02)
+	- Update bootstrap to stable version 5.3.0
+	- Fix hubloc confusion in mod rmagic
+	- Improved unseen forums notification
+	- Add workaround for friendica accept header bug for nginx
+	- Fix acl not set correctly in attach_store()
+	- Fix attachment name parsing
+	- Fix display issue in oembed video template
+	- content_import: fix crash in post handler
+	- pubcrawl: move addressing to separate function
+
+
+Hubzilla 8.4.1 (2023-05-20)
+	- Fix issue where accepting AP contacts would reset send stream permission
+	- Fix link to release page in upgrade info addon
+
+
+Hubzilla 8.4 (2023-05-17)
+	- Slightly rewrite Activity::store() to save a query
+	- Use act->objprop() in decode_note() to get activity values
+	- Make sure to re-discover a channel upon connecting if we have just an xchan but no hubloc
+	- Updated ES translations
+	- Updated RU translations
+	- Redesigned profile vcard to implement cover images
+	- Slightly improved discovery of AP quoted messages
+	- Updated bootstrap library
+	- Changed hashtag encoding to match mastodons expectations (issue #1750)
+	- Implement fedearted directory flags from trusted directory servers
+	- Use Activity::get_actor_hublocs() in Libzot::fetch_conversation() instead of custom queries
+	- Add the conversation endpoint
+	- Implement paginated fetch for zot requests in mod item
+	- Implement Zotconvo daemon to fetch conversations in the background
+	- Expose deleted channels to webfinger (otherwise we can not mark them deleted locally)
+	- Expire items in batches to prevent memory exhaustion
+	- Remove legacy zot compatibility tweaks
+	- Rewrite and simplify directory sync
+	- Improve potentially long running sql query finding the thread parent
+	- Implement option to override permissions for posts with mentions
+
+	Bugfixes
+	- Fix parent_mid and thr_parent not set correctly for response activities
+	- Fix query for postgres in attach_list_files()
+	- Fix item_url_replace() failing on arrays
+	- Fix OWA compatibility with friendica
+	- Fix regression in decoding the object type
+	- Fix xss vulnerability in justified gallery library
+	- Fix permissions not initialized after accepted follow request from AP
+	- Fix empty path passed to fopen()
+	- Fix % value in format_poll() not rounded
+	- Fix remove_all_xchan_resources() executed for local channels
+	- Fix likes not synced via libsync between clones
+
+	Addons
+	- Pubcrawl: fix xchan_url used as follow object instead of xchan_hash
+	- Channelreputation: moved to unmaintained repo due to duplication issues when storing data in pconfig
+	- Pubcrawl: add lemmy accept/reject follow quirks
+	- Diaspora: handle 4xx return codes in queue management
+	- Diaspora: fix fatal error in the case where parent_guid is set but empty
+	- Pubcrawl: if shared inbox delivery fails deliver to contacts instead of everybody we know from that site
+	- Articles: fix PHP error
+	- Diaspora: rewrite synced (from a clone) item author to primary
+	- Diaspora: improve reshare detection
+	- Diaspora: fix contact role not sets
+	- Pubcrawl: rewrite synced (from a clone) item author to primary
+	- Pubcrawl: if we receive a public message to the shared inbox deliver to following *and* directly addressed
+
+
+Hubzilla 8.2 (2023-03-19)
+	- Remove redundant untranslated htconfig templates
+	- Implement workaround for friendica image/attachment construct
+	- Implement notification exception for forum posts repeated by group actors
+	- Updated es translations
+	- Rewrite totp multifactor auth and move it from addon to core (ported from streams)
+	- Implement page load progress indicator for pwa standalone mode
+	- Improved mod HQ logic to save some DB lookups
+	- Default owner_xchan to sender in Activity::store() except for fetched items
+	- Improved contact search
+	- Do not force new thread if announce comes from group actor
+	- Disable oembed in notes app
+	- Improved queue handling
+	- Slightly restructure conv item head
+	- Implemet author images in HQ widget
+	- Implement author filter in HQ widget
+	- Add config option db_skip_locked_supported which should be set if the DB backend supports skip locked
+	- Updated simple pie library
+	- Updated smarty library
+	- Updated forkawesome library
+	- Updated fullcalendar library
+	- Implement native dark/light mode in favour of dark/light schema
+	- Updated bootstrap library
+
+	Bugfixes
+	- Fix channel calendar location html not interpreted - issue #1728
+	- Fix max oembed size check if content length header is an array
+	- Fix group actor wall posts turned into direct messages when posting article or card
+	- Fix non zot profile mentions using zrl tag
+	- Fix message filter > and < not decoded
+	- Fix deleted hublocs included in author lookup
+	- Fix some more PHP warnings
+	- Fix syntax for get_compund_property()
+	- Fix xchan_pubforum not set on actor updates
+	- Fix duplicate directory entries for xchans with bogus second primary entry
+	- Fix missing created timestamp in like activity - issue #1729
+	- Fix messages not signed when posting from non primary locations
+	- Fix pinned post regression
+	- Fix autocomplete items not selectable on mobile
+	- Fix hubloc confusion in Libzot::fetch_conversation()
+	- Fix valid duplicate notification entries deduplicated
+	- Fix ics file import
+	- Fix recipients deduplication in some places
+	- Fix hashtag encoded with href instead of id
+
+	Addons
+	- Pubcrawl: re-implement encrypted content which got lost in transition
+	- Pubcrawl: flag fetched items
+	- Totp: removed from addons in favor of implementation in core
+	- Pubcrawl: fix delivery issues for relayed activities
+	- Cart: fix missing bootstrap namespace
+	- Gallery: do not inject gallery code in webpages
+	- Pubcrawl: fix sql error if mentions string is empty
+	- Cart: fix paypal order hash size
+
+
+Hubzilla 8.0 (2023-01-13)
+	- Updated ru strings
+	- Implement ability to edit addon PDL with pdledit_gui and pdledit
+	- Implement queueworker auto sleep time
+	- Always add sys channel to receivers for public items
+	- Improved server sent events loop
+	- Exclude not tagable xchan networks handle_tag()
+	- Add mark html tag to the html2bbcode parser
+	- Embed or attach uploaded files depending on file type
+	- Changed logic to enable site only public stream - please revisit admin setting
+	- Updated nb-no strings
+	- Moved queueworker to core
+	- Provide a PDL file for mod invite and set the profile
+	- Disabled outdated context help
+	- Mark unseen items seen after a certain amount of time (default 90 days)
+	- Slightly restructure lib webfinger
+	- Improved redbasic dark schema
+	- Implement unseen thread and unseen items count in HQ widget
+	- In unseen items notifications just count the first 99 and display 99+ if there are more
+
+	Bugfixes
+	- Fix race conditions when processing multiple choice polls
+	- Fix affinity slider updates - issue #1714
+	- Fix issue where accounts created with a did2 could change their password
+	- Fix no message displayed if file not found in mod cloud
+	- Fix pubstream tagcloud unthreaded view
+	- Fix french invite template
+	- Fix comment permission issue in site only public stream
+	- Fix privacy tag not referring to latest hubloc entry
+	- Fix selected item not highlighted
+	- Fix poll issue when special chars and spaces are involved
+	- Fix prune_hub_reinstalls() to only care about zot6 hublocs
+	- Fix redis session PHP warnings
+	- Fix remote visitor photo uploads
+	- Fix mod shredwithme not showing any items
+	- Fix api_auth not referring to latest hubloc entry
+	- Fix category widget URLs
+	- Fix poller always polling broken feeds
+	- Fix sse event always triggered if public stream notification are off or the app is not installed
+	- Fix more PHP8 warnings
+	- Fix wrong variable in tag_deliver()
+	- Fix mod follow use trim() after punify() - issue #1698
+
+	Addons
+	- Cards: update widget info
+	- Articles: update widget info
+	- Wiki: update widget info
+	- Sse: filter apporvals
+	- Cards: use html_entity_decode before purify and html2plain for summary
+	- Articles: use html_entity_decode before purify and html2plain for summary
+	- Wiki: use html_entity_decode before purify and html2plain for summary
+	- Cards: fix comments not displayed if filter by category
+	- Articles: fix comments not displayed if filter by category
+	- Sse: respect the site public stream setting
+	- Queueworker: moved to core
+	- Pubcrawl: move db query out of foreach loop
+	- Pubcrawl: add mentions and thread owner to recipients for public items
+
+
+Hubzilla 7.8.7 (2022-12-03)
+	- Fix regression when adding feed contacts
+	- Fix regression new channel calendar event not created
+
+
+Hubzilla 7.8.6 (2022-11-14)
+	- Fix typo in boot.php
+
+
+Hubzilla 7.8.5 (2022-11-13)
+	- Fix outbound edit activity not of type update
+	- Fix mod display not falling through to fetch public item
+	- Fix more PHP warnings
+	- Fix regression in items_fetch() which resulted in empty atom feed
+	- Pubcrawl: cleanup and slightly restructre mod inbox
+
+
+Hubzilla 7.8.4 (2022-11-09)
+	- Fix new uuid created when editing a post
+
+
+Hubzilla 7.8.3 (2022-11-07)
+	- Fix regression where auto created directories were not created with public permissions
+	- Fix regression where pinned/featured state of apps was not displayed correctly
+
+
+Hubzilla 7.8.2 (2022-11-05)
+	- Pubcrawl: fix regression in inbox
+	- Fix display issue of shares coming from streams
+	- Throw a 404 if we could not determine which channel to load
+	- If we have a cached xchan/hubloc entry, make sure we fetch the latest
+	- Gallery: paint the background grid with css
+
+
+Hubzilla 7.8.1 (2022-10-26)
+	- Silence tons of PHP warnings in core
+	- Catch decryption failure edgecase in receiver
+	- Deal with conversation privacy mismatches in Activity::store()
+	- Composer libs updates
+	- Fix timeago strings not always translated
+	- Fix edit link regression in cards addon
+	- Fix edit link regression in article addon
+
+
+Hubzilla 7.8 (2022-10-10)
+	- Updated spanish translations
+	- Always update hubloc_updated timestamp if a hubloc is updated
+	- Do better with re-installs in various situations
+	- Update htconfig templates
+	- Emit an info if an item is deleted manually (issue #1691)
+	- Updated german translations
+	- Make Activity::actor_store() fetch the actor object if none is provided
+	- Check for various forms of as:Public for compatibility
+	- Cleanup dark schema
+	- Make use of CSS variables in redbasic
+	- Implement link hover color in redbasic
+	- Update to bootstrap version 5.2 which implements CSS variables
+
+	Bugfixes
+	- Fix stream filter for polls and events does not work anymore (issue #1694)
+	- Fix item menu display issue
+	- Fix HQ widget displaying superblocked items
+	- Fix mod poke
+	- Fix link to article or card not processed correctly if umlauts are involved (issue #1687)
+	- Fix mod siteinfo containing a dead link (issue #1690)
+	- Fix permission issue when displaying things (issue #1686).
+	- Fix check for existing profiles (issue #1688)
+	- Fix modal missing for conversation settings
+	- Fix missing closing div tag if there are no recent channel activities
+	- Fix issue where an unkown diaspora author was not imported if the comment arrived via a relayed activity
+	- Fix mangled xchan_url due to escape_tags()
+	- Fix the AP url not decode bin mod search
+	- Fix some php warnings
+
+	Addons
+	- Wiki: move create_missing_page() from widget to module
+	- Superblock: implement blocking for messages widget_item
+	- Diaspora: fix verification of relayed comments and likes
+	- Pubcrawl: make sure to store the attributedTo actor in announce activities
+	- Diaspora: fix issue where an unkown author was not imported if the comment arrived via a relayed activity
+
+
+Hubzilla 7.6.1 (2022-08-07)
+	Bugfixes
+	- Fix attachments displayed in visible response activities
+	- Fix wrong attribution in unseen like notifications
+
+	Addons
+	- Cards: fix widget not implemented via pdl file (requires re-install)
+	- Articles: fix widget not implemented via pdl file (requires re-install)
+	- Wiki: fix widget not implemented via pdl file (requires re-install)
+
+
+Hubzilla 7.6 (2022-07-26)
+	- Add filter rule to check for false condition
+	- Implement HQ dashboard to display recently created content for various modules
+	- Updated spanish translations
+	- Implement a max oembed size which defaults to 1MB
+	- Update composer libs
+
+	Addons
+	- Cart: do not attempt oembed
+	- Cards: implement channel_activities_widget for HQ dashboard
+	- Articles: implement channel_activities_widget for HQ dashboard
+	- Wiki: implement channel_activities_widget for HQ dashboard
+
+	Bugfixes
+	- Fix language filter filtering ambigous results
+	- Fix vcard-card background for dark schema
+	- Fix contact edit dialog not displayed in chrome browser
+	- Fix readability of category pills in dark schema
+	- Fix opengraph images inside zmg opening tag
+	- Fix oembed attempted for text previews
+
+
+Hubzilla 7.4 (2022-06-01)
+	- Updated russian translations
+	- Raise min PHP version to 8.0
+	- Rewrite inbound attachment handling
+	- Move photo flag feature from addon to core
+	- Adjust default production php logging
+	- Update composer libs
+	- Add a hidden config to enable the hs2019 http signature algorithm
+	- Allow starring of pubstream items
+	- Update spanish translations
+	- Add the title in forum post reshares if applicable
+	- Implement inbound pleroma reactions
+	- Add real name info to the navbar template data
+	- Add the updated date to the icon url so that other platforms will pick it up on change
+	- Move wiki from core to addon
+	- Move articles from core to addon
+	- Move cards from core to addon
+
+	Addons
+	- Rendezvous: add missing default value to sql schema
+	- Wiki: moved from core to addons
+	- Articles: moved from core to addons
+	- Cards: moved from core to addons
+	- Photoflag: moved to core
+	- Sse: fix issue where notifications were emited for created tasks
+
+	Bugfixes
+	- Fix issue where inbound activitypub items lost image descriptions (issue 1679)
+	- Fix issue where observer tags were not rendered correctly in event items (issue 1674)
+	- Fix forum posts leaked into network stream
+	- Fix tags not found in search if not logged in (issue 1677)
+	- Fix dislikes causing stuck notifications (issue 1676)
+
+
+Hubzilla 7.2.2 (2022-04-26)
+	- Fix item_verified not set due to data structure changes
+
+
+Hubzilla 7.2.1 (2022-04-25)
+	- Fix changing profile image from new member widget - issue #1671
+	- Fix regression with incoming poll answers from activitypub introduced in 7.2
+	- Fix addons not removed from the DB when removed from the filesystem
+	- Fix regression in attaching images for activitypub introduced in 7.2
+	- Move activitypub addressing from core to the pubcrawl addon
+	- Fix hub re-install issues
+	- Fediwordle: slightly improved algorithm
+
+
+Hubzilla 7.2 (2022-03-29)
+	- Streamline comment policy with downstream project
+	- Add new function is_local_url()
+	- Add helper function to escape URLs
+	- Add signing algorithm to zotinfo()
+	- Store signing algorithm in import_xchan()
+	- Use bootstrap-nightfall for redbasic:dark schema
+	- Add support for hs2019
+	- Remove unused function script_path()
+	- Move login and register button into the hamburger menu for small screens
+	- Collect accept headers in an array instead of a concatenated string
+	- Update composer libs
+	- Enhanced content filters
+	- Improve mod gprobe to also deal with URLs
+	- Adapt unseen forum posts query in mod network to new forum post style
+	- Remove deprecated widgets
+	- Add inbound support for quoteUrl to Lib/Activity
+	- Add widget descriptions
+	- Add a GUI for the PDL editor
+
+	Addons
+	- Pubcrawl: deprecate as.php in favor of core libs
+	- Pubcrawl: rewrite/modernize mod inbox
+	- Pubcrawl: reflect core enhanced content filter changes
+	- Diaspora: reflect core enhanced content filter changes
+	- Fediwordle: new addon - a distributed word game inspired by wordle
+	- Pubcrawl: streamline post_local hook with diaspora
+
+	Bugfixes
+	- Fix comments_closed date on posts where comments are disabled
+	- Fix open redirect via rpath query param (CVE-2022-27256)
+	- Fix cross-site scripting via rpath query param (CVE-2022-27258)
+	- Fix local file inclusion in redbasic theme (CVE-2022-27257)
+	- Fix baseurl for css and js
+	- Fix duplicate IDs in login form
+	- Fix unknown author not fetched if w2w comment arrives
+	- Fix thr_parent lost across edits
+
+
+Hubzilla 7.0.3 (2022-02-10)
+	- Allow to override the charset for the PDO connection string via $db_charset in .htconfig.php
+
+
 Hubzilla 7.0.2 (2022-02-09)
 	- Update french templates
 	- Add charset to the PDO connection strings

Zotlabs/Access/AccessList.php

@@ -2,6 +2,7 @@
 
 namespace Zotlabs\Access;
 
+
 /**
  * @brief AccessList class which represents individual content ACLs.
  *
@@ -17,29 +18,48 @@ class AccessList {
 	 * @brief Allow contacts
 	 * @var string
 	 */
-	private $allow_cid;
+	private ?string $allow_cid;
 	/**
 	 * @brief Allow groups
 	 * @var string
 	 */
-	private $allow_gid;
+	private ?string $allow_gid;
 	/**
 	 * @brief Deny contacts
 	 * @var string
 	 */
-	private $deny_cid;
+	private ?string $deny_cid;
 	/**
 	 * @brief Deny groups
 	 * @var string
 	 */
-	private $deny_gid;
+	private ?string $deny_gid;
 	/**
 	 * @brief Indicates if we are using the default constructor values or
 	 * values that have been set explicitly.
 	 * @var boolean
 	 */
-	private $explicit;
+	private bool $explicit;
 
+	/**
+	 * @brief Keys required by the constructor if the channel array is given.
+	 */
+	private const REQUIRED_KEYS_CONSTRUCTOR = [
+		'channel_allow_cid',
+		'channel_allow_gid',
+		'channel_deny_cid',
+		'channel_deny_gid'
+	];
+
+	/**
+	 * @brief Keys required by the set method.
+	 */
+	private const REQUIRED_KEYS_SET = [
+		'allow_cid',
+		'allow_gid',
+		'deny_cid',
+		'deny_gid'
+	];
 
 	/**
 	 * @brief Constructor for AccessList class.
@@ -53,8 +73,9 @@ class AccessList {
 	 *   * \e string \b channel_deny_cid => string of denied cids
 	 *   * \e string \b channel_deny_gid => string of denied gids
 	 */
-	function __construct($channel) {
+	function __construct(array $channel) {
 		if ($channel) {
+			$this->validate_input_array($channel, self::REQUIRED_KEYS_CONSTRUCTOR);
 			$this->allow_cid = $channel['channel_allow_cid'];
 			$this->allow_gid = $channel['channel_allow_gid'];
 			$this->deny_cid  = $channel['channel_deny_cid'];
@@ -70,13 +91,24 @@ class AccessList {
 		$this->explicit = false;
 	}
 
+	private function validate_input_array(array $arr, array $required_keys) : void {
+		$missing_keys = array_diff($required_keys, array_keys($arr));
+
+		if (!empty($missing_keys)) {
+			throw new \Exception(
+				'Invalid AccessList object: Expected array with keys: '
+				. implode(', ', $missing_keys)
+			);
+		}
+	}
+
 	/**
 	 * @brief Get if we are using the default constructor values
 	 * or values that have been set explicitly.
 	 *
 	 * @return boolean
 	 */
-	function get_explicit() {
+	function get_explicit() : bool {
 		return $this->explicit;
 	}
 
@@ -94,7 +126,9 @@ class AccessList {
 	 *   * \e string \b deny_gid  => string of denied gids
 	 * @param boolean $explicit (optional) default true
 	 */
-	function set($arr, $explicit = true) {
+	function set(array $arr, bool $explicit = true) : void {
+		$this->validate_input_array($arr, self::REQUIRED_KEYS_SET);
+
 		$this->allow_cid = $arr['allow_cid'];
 		$this->allow_gid = $arr['allow_gid'];
 		$this->deny_cid  = $arr['deny_cid'];
@@ -112,7 +146,7 @@ class AccessList {
 	 *   * \e string \b deny_cid  => string of denied cids
 	 *   * \e string \b deny_gid  => string of denied gids
 	 */
-	function get() {
+	function get() : array {
 		return [
 			'allow_cid' => $this->allow_cid,
 			'allow_gid' => $this->allow_gid,
@@ -138,7 +172,12 @@ class AccessList {
 	 *   * \e array|string \b group_deny    => array with gids or comma-seperated string
 	 * @param boolean $explicit (optional) default true
 	 */
-	function set_from_array($arr, $explicit = true) {
+	function set_from_array(array $arr, bool $explicit = true) : void {
+		$arr['contact_allow'] = $arr['contact_allow'] ?? [];
+		$arr['group_allow'] = $arr['group_allow'] ?? [];
+		$arr['contact_deny'] = $arr['contact_deny'] ?? [];
+		$arr['group_deny'] = $arr['group_deny'] ?? [];
+
 		$this->allow_cid = perms2str((is_array($arr['contact_allow']))
 			? $arr['contact_allow'] : explode(',', $arr['contact_allow']));
 		$this->allow_gid = perms2str((is_array($arr['group_allow']))
@@ -156,7 +195,7 @@ class AccessList {
 	 *
 	 * @return boolean Return true if any of allow_* deny_* values is set.
 	 */
-	function is_private() {
+	function is_private() : bool {
 		return (($this->allow_cid || $this->allow_gid || $this->deny_cid || $this->deny_gid) ? true : false);
 	}
 

Zotlabs/Daemon/Addon.php

@@ -7,6 +7,7 @@ class Addon {
 	static public function run($argc, $argv) {
 
 		call_hooks('daemon_addon', $argv);
+		return;
 
 	}
 

Zotlabs/Daemon/Cache_embeds.php

@@ -19,8 +19,10 @@ class Cache_embeds {
 		$item = $c[0];
 
 		// bbcode conversion by default processes embeds that aren't already cached.
-		// Ignore the returned html output. 
+		// Ignore the returned html output.
 
 		bbcode($item['body']);
+
+		return;
 	}
 }

Zotlabs/Daemon/Cache_query.php

@@ -24,7 +24,7 @@ class Cache_query {
 
 		array_shift($argv);
 		array_shift($argv);
-		
+
 		$arr = json_decode(base64_decode($argv[0]), true);
 
 		$r = call_user_func_array('q', $arr);
@@ -32,5 +32,7 @@ class Cache_query {
 			Cache::set($key, serialize($r));
 
 		del_config('procid', $key);
+
+		return;
 	}
 }

Zotlabs/Daemon/Checksites.php

@@ -10,6 +10,9 @@ class Checksites {
 
 		logger('checksites: start');
 
+		$site_id = '';
+		$sql_options = '';
+
 		if (($argc > 1) && ($argv[1]))
 			$site_id = $argv[1];
 

Zotlabs/Daemon/Cli_suggest.php

@@ -9,6 +9,7 @@ class Cli_suggest {
 	static public function run($argc,$argv) {
 
 		update_suggestions();
+		return;
 
 	}
 }

Zotlabs/Daemon/Convo.php

@@ -30,6 +30,7 @@ class Convo {
 			intval($channel_id),
 			dbesc($contact_hash)
 		);
+
 		if (!$r) {
 			return;
 		}
@@ -40,19 +41,26 @@ class Convo {
 
 		$messages = $obj->get();
 
-		if ($messages) {
-			foreach ($messages as $message) {
-				if (is_string($message)) {
-					$message = Activity::fetch($message, $channel);
-				}
-				// set client flag because comments will probably just be objects and not full blown activities
-				// and that lets us use implied_create
-				$AS = new ActivityStreams($message);
-				if ($AS->is_valid() && is_array($AS->obj)) {
-					$item = Activity::decode_note($AS);
-					Activity::store($channel, $contact['abook_xchan'], $AS, $item);
-				}
+		if (!$messages) {
+			return;
+		}
+
+		foreach ($messages as $message) {
+			if (is_string($message)) {
+				$message = Activity::fetch($message, $channel);
+			}
+
+			// set client flag because comments will probably just be objects and not full blown activities
+			// and that lets us use implied_create
+			$AS = new ActivityStreams($message);
+			if ($AS->is_valid() && is_array($AS->obj)) {
+				$item = Activity::decode_note($AS);
+				$item['item_fetched'] = true;
+				Activity::store($channel, $contact['abook_xchan'], $AS, $item);
 			}
 		}
+
+		return;
+
 	}
 }

Zotlabs/Daemon/Cron.php

@@ -3,6 +3,7 @@
 namespace Zotlabs\Daemon;
 
 use Zotlabs\Lib\Libsync;
+use Zotlabs\Lib\Libzotdir;
 
 class Cron {
 
@@ -19,6 +20,7 @@ class Cron {
 			}
 		}
 
+/*
 		// Check for a lockfile.  If it exists, but is over an hour old, it's stale.  Ignore it.
 		$lockfile = 'store/[data]/cron';
 		if ((file_exists($lockfile)) && (filemtime($lockfile) > (time() - 3600))
@@ -30,9 +32,21 @@ class Cron {
 		// Create a lockfile.  Needs two vars, but $x doesn't need to contain anything.
 		$x = '';
 		file_put_contents($lockfile, $x);
+*/
 
 		logger('cron: start');
 
+		// If this is a directory server, request a sync with an upstream
+		// directory at least once a day, up to once every poll interval.
+		// Pull remote changes and push local changes.
+		// potential issue: how do we keep from creating an endless update loop?
+
+		$dirmode = get_config('system', 'directory_mode');
+
+		if ($dirmode == DIRECTORY_MODE_SECONDARY || $dirmode == DIRECTORY_MODE_PRIMARY) {
+			Libzotdir::sync_directories($dirmode);
+		}
+
 		// run queue delivery process in the background
 
 		Master::Summon(array('Queue'));
@@ -50,7 +64,7 @@ class Cron {
 		require_once('include/account.php');
 		remove_expired_registrations();
 
-		$interval = get_config('system', 'delivery_interval', 3);
+		$interval = get_config('queueworker', 'queue_interval', 500000);
 
 		// expire any expired items
 
@@ -65,8 +79,10 @@ class Cron {
 				if ($rr['item_wall']) {
 					// The notifier isn't normally invoked unless item_drop is interactive.
 					Master::Summon(['Notifier', 'drop', $rr['id']]);
-					if ($interval)
-						@time_sleep_until(microtime(true) + (float)$interval);
+
+					if ($interval) {
+						usleep($interval);
+					}
 				}
 			}
 		}
@@ -96,8 +112,10 @@ class Cron {
 		if ($r) {
 			foreach ($r as $rr) {
 				Master::Summon(array('Directory', $rr['channel_id'], 'force'));
-				if ($interval)
-					@time_sleep_until(microtime(true) + (float)$interval);
+
+				if ($interval) {
+					usleep($interval);
+				}
 			}
 		}
 
@@ -151,8 +169,10 @@ class Cron {
 						);
 					}
 					Master::Summon(array('Notifier', 'wall-new', $rr['id']));
-					if ($interval)
-						@time_sleep_until(microtime(true) + (float)$interval);
+
+					if ($interval) {
+						usleep($interval);
+					}
 				}
 			}
 		}
@@ -203,10 +223,10 @@ class Cron {
 		}
 
 
-		// pull in some public posts
+		// pull in some public posts if allowed
 
-		$disable_discover_tab = get_config('system', 'disable_discover_tab') || get_config('system', 'disable_discover_tab') === false;
-		if (!$disable_discover_tab)
+		$disable_externals = get_config('system', 'disable_discover_tab') || get_config('system', 'disable_discover_tab') === false || get_config('system', 'site_firehose');
+		if (!$disable_externals)
 			Master::Summon(['Externals']);
 
 		$restart = false;
@@ -228,7 +248,7 @@ class Cron {
 		set_config('system', 'lastcron', datetime_convert());
 
 		//All done - clear the lockfile
-		@unlink($lockfile);
+		//@unlink($lockfile);
 
 		return;
 	}

Zotlabs/Daemon/Cron_daily.php

@@ -49,6 +49,22 @@ class Cron_daily {
 			dbesc('sse_id.%')
 		);
 
+		// Mark items seen after X days (default 90)
+
+		$r = dbq("select channel_id from channel where channel_removed = 0");
+		if ($r) {
+			foreach ($r as $rr) {
+				$mark_seen_days = get_pconfig($rr['channel_id'], 'system', 'mark_seen_days', 90);
+				q("UPDATE item SET item_unseen = 0 WHERE
+					uid = %d AND item_unseen = 1
+					AND created < %s - INTERVAL %s",
+					intval($rr['channel_id']),
+					db_utcnow(),
+					db_quoteinterval($mark_seen_days . ' DAY')
+				);
+			}
+		}
+
 		// Clean up emdedded content cache
 		q("DELETE FROM cache WHERE updated < %s - INTERVAL %s",
 			db_utcnow(),
@@ -78,17 +94,6 @@ class Cron_daily {
 		// expire any expired accounts
 		downgrade_accounts();
 
-		// If this is a directory server, request a sync with an upstream
-		// directory at least once a day, up to once every poll interval.
-		// Pull remote changes and push local changes.
-		// potential issue: how do we keep from creating an endless update loop?
-
-		$dirmode = get_config('system', 'directory_mode');
-
-		if ($dirmode == DIRECTORY_MODE_SECONDARY || $dirmode == DIRECTORY_MODE_PRIMARY) {
-			Libzotdir::sync_directories($dirmode);
-		}
-
 		Master::Summon(array('Expire'));
 		Master::Summon(array('Cli_suggest'));
 
@@ -103,5 +108,7 @@ class Cron_daily {
 		/**
 		 * End Cron Daily
 		 */
+
+		 return;
 	}
 }

Zotlabs/Daemon/Cron_weekly.php

@@ -22,12 +22,12 @@ class Cron_weekly {
 
 		mark_orphan_hubsxchans();
 
-		// Find channels that were removed in the last three weeks, but 
+		// Find channels that were removed in the last three weeks, but
 		// haven't been finally cleaned up. These should be older than 10
-		// days to ensure that "purgeall" messages have gone out or bounced 
-		// or timed out. 
+		// days to ensure that "purgeall" messages have gone out or bounced
+		// or timed out.
 
-		$r = q("select channel_id from channel where channel_removed = 1 and 
+		$r = q("select channel_id from channel where channel_removed = 1 and
 			channel_deleted >  %s - INTERVAL %s and channel_deleted < %s - INTERVAL %s",
 			db_utcnow(), db_quoteinterval('21 DAY'),
 			db_utcnow(), db_quoteinterval('10 DAY')
@@ -59,5 +59,6 @@ class Cron_weekly {
 		 * End Cron Weekly
 		 */
 
+		return;
 	}
-}
+}

Zotlabs/Daemon/Deliver.php

@@ -28,6 +28,8 @@ class Deliver {
 
 		}
 
+		return;
+
 	}
 
 }

Zotlabs/Daemon/Deliver_hooks.php

@@ -12,8 +12,12 @@ class Deliver_hooks {
 		$r = q("select * from item where id = '%d'",
 			intval($argv[1])
 		);
-		if ($r)
+
+		if ($r) {
 			call_hooks('notifier_normal', $r[0]);
+		}
+
+		return;
 
 	}
 }

Zotlabs/Daemon/Directory.php

@@ -97,5 +97,7 @@ class Directory {
 		if ($pushall) {
 			Master::Summon(array('Notifier', 'refresh_all', $channel['channel_id']));
 		}
+
+		return;
 	}
 }

Zotlabs/Daemon/Expire.php

@@ -32,7 +32,6 @@ class Expire {
 		}
 
 		// physically remove anything that has been deleted for more than two months
-		/** @FIXME - this is a wretchedly inefficient query */
 
 		q("delete from item where item_pending_remove = 1 and changed < %s - INTERVAL %s",
 			db_utcnow(),
@@ -59,8 +58,8 @@ class Expire {
 					continue;
 
 				// service class default (if non-zero) over-rides the site default
-
 				$service_class_expire = service_class_fetch($rr['channel_id'], 'expire_days');
+
 				if (intval($service_class_expire))
 					$channel_expire = $service_class_expire;
 				else
@@ -85,7 +84,6 @@ class Expire {
 
 			// this should probably just fetch the channel_expire_days from the sys channel,
 			// but there's no convenient way to set it.
-
 			$expire_days = get_config('system', 'sys_expire_days');
 			if ($expire_days === false)
 				$expire_days = 30;
@@ -96,12 +94,15 @@ class Expire {
 
 			logger('Expire: sys interval: ' . $expire_days, LOGGER_DEBUG);
 
-			if ($expire_days)
+			if ($expire_days) {
 				item_expire($x['channel_id'], $expire_days, $commented_days);
+			}
 
 			logger('Expire: sys: done', LOGGER_DEBUG);
 		}
 
 		del_config('procid', 'expire');
+
+		return;
 	}
 }

Zotlabs/Daemon/Externals.php

@@ -19,6 +19,7 @@ class Externals {
 		$importer = get_sys_channel();
 		$total    = 0;
 		$attempts = 0;
+		$url = '';
 
 		logger('externals: startup', LOGGER_DEBUG);
 
@@ -67,9 +68,8 @@ class Externals {
 					datetime_convert('UTC', 'UTC', 'now - 30 days')
 				);
 
-				$contact = $r[0];
-
-				if ($contact) {
+				if ($r) {
+					$contact = $r[0];
 					$url = $contact['hubloc_id_url'];
 				}
 			}
@@ -133,7 +133,7 @@ class Externals {
 								continue;
 							}
 
-							$obj_id = isset($message['object']['id']) ?? $message['object'];
+							$obj_id = $message['object']['id'] ?? $message['object'];
 
 							Libzot::fetch_conversation($importer, $obj_id);
 							$total++;
@@ -143,7 +143,8 @@ class Externals {
 						$AS = new ActivityStreams($message);
 						if ($AS->is_valid() && is_array($AS->obj)) {
 							$item = Activity::decode_note($AS);
-							Activity::store($importer, $contact['abook_xchan'], $AS, $item);
+							$item['item_fetched'] = true;
+							Activity::store($importer, $contact['hubloc_hash'], $AS, $item);
 							$total++;
 						}
 					}

Zotlabs/Daemon/Gprobe.php

@@ -15,21 +15,32 @@ class Gprobe {
 			return;
 
 		$url = hex2bin($argv[1]);
+		$is_webbie = false;
+		$r = null;
 
-		if (!strpos($url, '@'))
-			return;
+		if (filter_var($url, FILTER_VALIDATE_EMAIL)) {
+			$is_webbie = true;
 
-		$r = q("select * from hubloc where hubloc_addr = '%s' and hubloc_network = 'zot6' limit 1",
-			dbesc($url)
-		);
+			$r = q("select * from hubloc where hubloc_addr = '%s' and hubloc_network = 'zot6' limit 1",
+				dbesc($url)
+			);
+		}
+		elseif (filter_var($url, FILTER_VALIDATE_URL)) {
+			$r = q("select * from hubloc where hubloc_id_url = '%s' and hubloc_network = 'zot6' limit 1",
+				dbesc($url)
+			);
+		}
 
 		if (!$r) {
-			$href = Webfinger::zot_url(punify($url));
-			if ($href) {
-				$zf = Zotfinger::exec($href, null);
+			if ($is_webbie) {
+				$url = Webfinger::zot_url(punify($url));
 			}
-			if (is_array($zf) && array_path_exists('signature/signer', $zf) && $zf['signature']['signer'] === $href && intval($zf['signature']['header_valid'])) {
-				Libzot::import_xchan($zf['data']);
+
+			if ($url) {
+				$zf = Zotfinger::exec($url, null);
+				if (is_array($zf) && array_path_exists('signature/signer', $zf) && $zf['signature']['signer'] === $url && intval($zf['signature']['header_valid'])) {
+					Libzot::import_xchan($zf['data']);
+				}
 			}
 		}
 

Zotlabs/Daemon/Importdoc.php

@@ -11,6 +11,8 @@ class Importdoc {
 
 		self::update_docs_dir('doc/*');
 
+		return;
+
 	}
 
 	static public function update_docs_dir($s) {

Zotlabs/Daemon/Master.php

@@ -2,6 +2,8 @@
 
 namespace Zotlabs\Daemon;
 
+use Zotlabs\Lib\QueueWorker;
+
 if (array_search(__file__, get_included_files()) === 0) {
 	require_once('include/cli_startup.php');
 	array_shift($argv);
@@ -9,6 +11,7 @@ if (array_search(__file__, get_included_files()) === 0) {
 
 	if ($argc)
 		Master::Release($argc, $argv);
+
 	return;
 }
 
@@ -16,6 +19,10 @@ if (array_search(__file__, get_included_files()) === 0) {
 class Master {
 
 	static public function Summon($arr) {
+
+		QueueWorker::Summon($arr);
+		return;
+/*
 		$hookinfo = [
 			'argv' => $arr
 		];
@@ -32,11 +39,15 @@ class Master {
 
 		$phpbin = get_config('system', 'phpbin', 'php');
 		proc_run($phpbin, 'Zotlabs/Daemon/Master.php', $arr);
+*/
 	}
 
 	static public function Release($argc, $argv) {
 		cli_startup();
 
+		QueueWorker::Release($argv);
+		return;
+/*
 		$hookinfo = [
 			'argv' => $argv
 		];
@@ -54,5 +65,6 @@ class Master {
 		logger('Master: release: ' . json_encode($argv), LOGGER_ALL, LOG_DEBUG);
 		$cls = '\\Zotlabs\\Daemon\\' . $argv[0];
 		$cls::run($argc, $argv);
+*/
 	}
 }

Zotlabs/Daemon/Notifier.php

@@ -79,10 +79,7 @@ class Notifier {
 	static public $encoded_item = null;
 	static public $channel = null;
 	static public $private = false;
-	// $fragment can contain additional info to omit de-duplication in the queueworker.
-	// E.g. if an item is updated many times in a row from different sources (multiple vote updates) the
-	// update source mid or a timestamp or random string can be added.
-	static public $fragment = null;
+
 
 	static public function run($argc, $argv) {
 
@@ -107,10 +104,15 @@ class Notifier {
 		self::$encoded_item = null;
 		self::$channel      = null;
 		self::$private      = false;
-		self::$fragment     = null;
 
 		$sys         = get_sys_channel();
 		$normal_mode = true;
+		$upstream = false;
+		$uplink = false;
+		$target_item = null;
+		$parent_item = null;
+		$top_level_post = false;
+		$relay_to_owner = false;
 
 		if ($cmd === 'keychange') {
 			self::$channel = channelx_by_n($item_id);
@@ -227,8 +229,6 @@ class Notifier {
 
 			// Fetch the target item
 
-			self::$fragment = $argv[3] ?? '';
-
 			$r = q("SELECT * FROM item WHERE id = %d AND parent != 0",
 				intval($item_id)
 			);
@@ -300,6 +300,11 @@ class Notifier {
 				return;
 			}
 
+			if ($target_item['verb'] === ACTIVITY_SHARE) {
+				// Provide correct representation across the wire. Internally this is treated as a comment.
+				$target_item['parent_mid'] = $target_item['thr_parent'] = $target_item['mid'];
+			}
+
 			if ($target_item['mid'] === $target_item['parent_mid']) {
 				$parent_item    = $target_item;
 				$top_level_post = true;
@@ -360,8 +365,6 @@ class Notifier {
 			// the hostname in the message_id and provides a second (fallback) opinion.
 
 			$relay_to_owner = (!$top_level_post && intval($target_item['item_origin']) && comment_local_origin($target_item));
-			$uplink         = false;
-			$upstream       = false;
 
 			// $cmd === 'relay' indicates the owner is sending it to the original recipients
 			// don't allow the item in the relay command to relay to owner under any circumstances, it will loop
@@ -379,7 +382,7 @@ class Notifier {
 
 			if (($relay_to_owner || $uplink) && ($cmd !== 'relay')) {
 				logger('notifier: followup relay', LOGGER_DEBUG);
-				$sendto            = (($uplink) ? $parent_item['source_xchan'] : $parent_item['owner_xchan']);
+				$sendto            = (($uplink) ? $parent_item['source_xchan'] : (($parent_item['verb'] === ACTIVITY_SHARE) ? $parent_item['author_xchan'] : $parent_item['owner_xchan']));
 				self::$recipients  = [$sendto];
 				self::$private     = true;
 				$upstream          = true;
@@ -411,9 +414,7 @@ class Notifier {
 				self::$private    = false;
 				self::$recipients = collect_recipients($parent_item, self::$private);
 
-				// FIXME add any additional recipients such as mentions, etc.
-
-				if ($top_level_post) {
+				if ($top_level_post && intval($target_item['item_wall'])) {
 					// remove clones who will receive the post via sync
 					self::$recipients = array_values(array_diff(self::$recipients, [$target_item['owner_xchan']]));
 				}
@@ -505,6 +506,7 @@ class Notifier {
 		// public posts won't make it to the local public stream unless there's a recipient on this site.
 		// This code block sees if it's a public post and localhost is missing, and if so adds an entry for the local sys channel to the $hubs list
 
+		/* sys channel is now added in collect recipients
 		if (!self::$private) {
 			$found_localhost = false;
 			if ($hubs) {
@@ -525,6 +527,7 @@ class Notifier {
 				}
 			}
 		}
+		*/
 
 		if (!$hubs) {
 			logger('notifier: no hubs', LOGGER_NORMAL, LOG_NOTICE);
@@ -647,13 +650,14 @@ class Notifier {
 			// This shouldn't produce false positives on comment boosts that were generated on other platforms
 			// because we won't be delivering them.
 
-			if (isset($target_item) && isset($target_item['verb']) && $target_item['verb'] === 'Announce' && $target_item['author_xchan'] === $target_item['owner_xchan'] && ! intval($target_item['item_thread_top'])) {
+			if (isset($target_item['verb']) && $target_item['verb'] === 'Announce' && $target_item['author_xchan'] === $target_item['owner_xchan'] && ! intval($target_item['item_thread_top'])) {
 				continue;
 			}
 
 			$hash = new_uuid();
 
-			$env = (($hub_env && $hub_env[$hub['hubloc_site_id']]) ? $hub_env[$hub['hubloc_site_id']] : '');
+			$env = $hub_env[$hub['hubloc_site_id']] ?? '';
+
 			if ((self::$private) && (!$env)) {
 				continue;
 			}
@@ -690,11 +694,11 @@ class Notifier {
 
 		}
 
-		if ($normal_mode) {
+		if ($normal_mode && is_array($target_item)) {
 			// This wastes a process if there are no delivery hooks configured, so check this before launching the new process
 			$x = q("select * from hook where hook = 'notifier_normal'");
 			if ($x) {
-				Master::Summon(['Deliver_hooks', $target_item['id'], self::$fragment]);
+				Master::Summon(['Deliver_hooks', $target_item['id']]);
 			}
 		}
 

Zotlabs/Daemon/Onedirsync.php

@@ -15,61 +15,45 @@ class Onedirsync {
 			$update_id = intval($argv[1]);
 
 		if (!$update_id) {
-			logger('onedirsync: no update');
+			logger('onedirsync: no update id');
 			return;
 		}
 
-		$r = q("select * from updates where ud_id = %d limit 1",
+		$r = q("select * from updates where ud_id = %d",
 			intval($update_id)
 		);
 
-		if (!$r)
-			return;
-
-		if (($r[0]['ud_flags'] & UPDATE_FLAGS_UPDATED) || (!$r[0]['ud_addr']))
-			return;
-
-		// Have we probed this channel more recently than the other directory server
-		// (where we received this update from) ?
-		// If we have, we don't need to do anything except mark any older entries updated
-
-		$x = q("select * from updates where ud_addr = '%s' and ud_date > '%s' and ( ud_flags & %d )>0 order by ud_date desc limit 1",
-			dbesc($r[0]['ud_addr']),
-			dbesc($r[0]['ud_date']),
-			intval(UPDATE_FLAGS_UPDATED)
-		);
-		if ($x) {
-			q("update updates set ud_flags = ( ud_flags | %d ) where ud_addr = '%s' and ( ud_flags & %d ) = 0 and ud_date != '%s'",
-				intval(UPDATE_FLAGS_UPDATED),
-				dbesc($r[0]['ud_addr']),
-				intval(UPDATE_FLAGS_UPDATED),
-				dbesc($x[0]['ud_date'])
-			);
+		if (!$r) {
+			logger('onedirsync: update id not found');
 			return;
 		}
 
 		// ignore doing an update if this ud_addr refers to a known dead hubloc
 
-		$h = q("select * from hubloc where hubloc_addr = '%s'",
+		$h = q("select * from hubloc where hubloc_id_url = '%s' order by hubloc_id desc",
 			dbesc($r[0]['ud_addr']),
 		);
 
 		$h = Libzot::zot_record_preferred($h);
 
 		if (($h) && (($h['hubloc_status'] & HUBLOC_OFFLINE) || $h['hubloc_deleted'] || $h['hubloc_error'])) {
-			q("update updates set ud_flags = ( ud_flags | %d ) where ud_addr = '%s' and ( ud_flags & %d ) = 0 ",
-				intval(UPDATE_FLAGS_DELETED),
-				dbesc($r[0]['ud_addr']),
-				intval(UPDATE_FLAGS_UPDATED)
-			);
-			return;
+
+			// 2023-04-12: Try to update anyway since the info is not always correct
+			// This might change after all directory servers run the new code.
+
+			// q("update updates set ud_flags = 9 where ud_hash = '%s' and ud_flags != 9",
+			//	dbesc($r[0]['ud_hash'])
+			//);
+
+			// return;
 		}
 
 		// we might have to pull this out some day, but for now update_directory_entry()
 		// runs zot_finger() and is kind of zot specific
 
-		if ($h && $h['hubloc_network'] !== 'zot6')
+		if ($h && $h['hubloc_network'] !== 'zot6') {
 			return;
+		}
 
 		Libzotdir::update_directory_entry($r[0]);
 

Zotlabs/Daemon/Onepoll.php

@@ -30,14 +30,10 @@ class Onepoll {
 			$sql_extra = ' and abook_feed = 0 ';
 		}
 
-		$contacts = q("SELECT abook.*, xchan.*, account.*
-			FROM abook LEFT JOIN account on abook_account = account_id left join xchan on xchan_hash = abook_xchan
-			where abook_id = %d $sql_extra
-			and abook_pending = 0 and abook_archived = 0 and abook_blocked = 0 and abook_ignored = 0
-			AND (( account_flags = %d ) OR ( account_flags = %d )) limit 1",
-			intval($contact_id),
-			intval(ACCOUNT_OK),
-			intval(ACCOUNT_UNVERIFIED)
+		$contacts = q("SELECT abook.*, xchan.* FROM abook
+			LEFT JOIN xchan ON xchan_hash = abook_xchan
+			WHERE abook_id = %d",
+			intval($contact_id)
 		);
 
 		if (!$contacts) {
@@ -46,6 +42,7 @@ class Onepoll {
 		}
 
 		$contact      = $contacts[0];
+
 		$importer_uid = $contact['abook_channel'];
 
 		$importer = channelx_by_n($importer_uid);
@@ -53,7 +50,7 @@ class Onepoll {
 		if (!$importer)
 			return;
 
-		logger("onepoll: poll: ({$contact['id']}) IMPORTER: {$importer['xchan_name']}, CONTACT: {$contact['xchan_name']}");
+		logger("onepoll: poll: ($contact_id) IMPORTER: {$importer['xchan_name']}, CONTACT: {$contact['xchan_name']}");
 
 		$last_update = ((($contact['abook_updated'] === $contact['abook_created']) || ($contact['abook_updated'] <= NULL_DATE))
 			? datetime_convert('UTC', 'UTC', 'now - 7 days')
@@ -63,12 +60,21 @@ class Onepoll {
 		if ($contact['xchan_network'] === 'rss') {
 			logger('onepoll: processing feed ' . $contact['xchan_name'], LOGGER_DEBUG);
 			$alive = handle_feed($importer['channel_id'], $contact_id, $contact['xchan_hash']);
-			if ($alive) {
-				q("update abook set abook_connected = '%s' where abook_id = %d",
+
+			if (!$alive) {
+				q("update abook set abook_updated = '%s' where abook_id = %d",
 					dbesc(datetime_convert()),
 					intval($contact['abook_id'])
 				);
+				return;
 			}
+
+			q("update abook set abook_updated = '%s', abook_connected = '%s' where abook_id = %d",
+				dbesc(datetime_convert()),
+				dbesc(datetime_convert()),
+				intval($contact['abook_id'])
+			);
+
 			return;
 		}
 
@@ -162,6 +168,7 @@ class Onepoll {
 							$AS = new ActivityStreams($message);
 							if ($AS->is_valid() && is_array($AS->obj)) {
 								$item = Activity::decode_note($AS);
+								$item['item_fetched'] = true;
 								Activity::store($importer, $contact['abook_xchan'], $AS, $item);
 							}
 						}

Zotlabs/Daemon/Poller.php

@@ -17,21 +17,7 @@ class Poller {
 			}
 		}
 
-		$interval = intval(get_config('system', 'poll_interval'));
-		if (!$interval)
-			$interval = ((get_config('system', 'delivery_interval') === false) ? 3 : intval(get_config('system', 'delivery_interval')));
-
-		// Check for a lockfile.  If it exists, but is over an hour old, it's stale.  Ignore it.
-		$lockfile = 'store/[data]/poller';
-		if ((file_exists($lockfile)) && (filemtime($lockfile) > (time() - 3600))
-			&& (!get_config('system', 'override_poll_lockfile'))) {
-			logger("poller: Already running");
-			return;
-		}
-
-		// Create a lockfile.  Needs two vars, but $x doesn't need to contain anything.
-		$x = '';
-		file_put_contents($lockfile, $x);
+		$interval = get_config('queueworker', 'queue_interval', 500000);
 
 		logger('poller: start');
 
@@ -63,6 +49,11 @@ class Poller {
 			: ''
 		);
 
+		$allow_feeds = get_config('system', 'feed_contacts');
+		if(!$allow_feeds) {
+			$sql_extra .= ' and abook_feed = 0 ';
+		}
+
 		$randfunc = db_getfunc('RAND');
 
 		$contacts = q("SELECT abook.abook_updated, abook.abook_connected, abook.abook_feed,
@@ -72,7 +63,7 @@ class Poller {
 			account.account_lastlog, account.account_flags
 			FROM abook LEFT JOIN xchan on abook_xchan = xchan_hash
 			LEFT JOIN account on abook_account = account_id
-			where abook_self = 0
+			where abook_self = 0 and abook_pending = 0 and abook_archived = 0 and abook_blocked = 0 and abook_ignored = 0
 			$sql_extra
 			AND (( account_flags = %d ) OR ( account_flags = %d )) $abandon_sql ORDER BY $randfunc",
 			intval(ACCOUNT_OK),
@@ -93,13 +84,24 @@ class Poller {
 						$min = intval(get_config('system', 'minimum_feedcheck_minutes'));
 					if (!$min)
 						$min = 60;
-					$x = datetime_convert('UTC', 'UTC', "now - $min minutes");
-					if ($c < $x) {
-						Master::Summon(['Onepoll', $contact['abook_id']]);
-						if ($interval)
-							@time_sleep_until(microtime(true) + (float)$interval);
+
+					if ($t !== $c) {
+						// if the last fetch failed only attempt fetch once a day
+						$min = 60 * 24;
 					}
+
+					$x = datetime_convert('UTC', 'UTC', "now - $min minutes");
+
+					if ($t < $x) {
+						Master::Summon(['Onepoll', $contact['abook_id']]);
+
+						if ($interval) {
+							usleep($interval);
+						}
+					}
+
 					continue;
+
 				}
 
 				if ($contact['xchan_network'] !== 'zot6')
@@ -158,20 +160,22 @@ class Poller {
 					continue;
 
 				Master::Summon(['Onepoll', $contact['abook_id']]);
-				if ($interval)
-					@time_sleep_until(microtime(true) + (float)$interval);
 
+				if ($interval) {
+					usleep($interval);
+				}
 			}
 		}
 
 		$dirmode = intval(get_config('system', 'directory_mode'));
 
 		if ($dirmode == DIRECTORY_MODE_SECONDARY || $dirmode == DIRECTORY_MODE_PRIMARY) {
-			$r = q("SELECT u.ud_addr, u.ud_id, u.ud_last FROM updates AS u INNER JOIN (SELECT ud_addr, max(ud_id) AS ud_id FROM updates WHERE ( ud_flags & %d ) = 0 AND ud_addr != '' AND ( ud_last <= '%s' OR ud_last > %s - INTERVAL %s ) GROUP BY ud_addr) AS s ON s.ud_id = u.ud_id ",
-				intval(UPDATE_FLAGS_UPDATED),
+			$r = q("SELECT * FROM updates WHERE ud_update = 1 AND (ud_last = '%s' OR ud_last > %s - INTERVAL %s)",
 				dbesc(NULL_DATE),
-				db_utcnow(), db_quoteinterval('7 DAY')
+				db_utcnow(),
+				db_quoteinterval('7 DAY')
 			);
+
 			if ($r) {
 				foreach ($r as $rr) {
 
@@ -181,19 +185,18 @@ class Poller {
 					if ($rr['ud_last'] > NULL_DATE)
 						if ($rr['ud_last'] > datetime_convert('UTC', 'UTC', 'now - 1 day'))
 							continue;
+
 					Master::Summon(['Onedirsync', $rr['ud_id']]);
-					if ($interval)
-						@time_sleep_until(microtime(true) + (float)$interval);
+
+					if ($interval) {
+						usleep($interval);
+					}
 				}
 			}
 		}
 
 		set_config('system', 'lastpoll', datetime_convert());
 
-		//All done - clear the lockfile
-
-		@unlink($lockfile);
-
 		return;
 	}
 }

Zotlabs/Daemon/Queue.php

@@ -7,77 +7,59 @@ use Zotlabs\Lib\Queue as LibQueue;
 class Queue {
 
 	static public function run($argc, $argv) {
-
-		require_once('include/items.php');
-		require_once('include/bbcode.php');
-
-		if ($argc > 1)
-			$queue_id = $argv[1];
-		else
-			$queue_id = EMPTY_STR;
+		$queue_id = ($argc > 1) ? $argv[1] : '';
 
 		logger('queue: start');
 
 		// delete all queue items more than 3 days old
 		// but first mark these sites dead if we haven't heard from them in a month
 
-		$r = q("select outq_posturl from outq where outq_created < %s - INTERVAL %s",
-			db_utcnow(), db_quoteinterval('3 DAY')
+		$oldqItems = q("select outq_posturl, outq_hash from outq where outq_created < %s - INTERVAL %s",
+			db_utcnow(),
+			db_quoteinterval('3 DAY')
 		);
-		if ($r) {
-			foreach ($r as $rr) {
-				$h       = parse_url($rr['outq_posturl']);
-				$desturl = $h['scheme'] . '://' . $h['host'] . (isset($h['port']) ? ':' . $h['port'] : '');
+
+		if ($oldqItems) {
+			foreach ($oldqItems as $qItem) {
+				$h = parse_url($qItem['outq_posturl']);
+				$site_url = $h['scheme'] . '://' . $h['host'] . ((!empty($h['port'])) ? ':' . $h['port'] : '');
 				q("update site set site_dead = 1 where site_dead = 0 and site_url = '%s' and site_update < %s - INTERVAL %s",
-					dbesc($desturl),
-					db_utcnow(), db_quoteinterval('1 MONTH')
+					dbesc($site_url),
+					db_utcnow(),
+					db_quoteinterval('1 MONTH')
 				);
 			}
+
+			$old_hashes = ids_to_querystr($oldqItems, 'outq_hash', true);
+
+			logger('Removing ' . count($oldqItems) . ' old queue entries');
+			dbq("DELETE FROM outq WHERE outq_hash IN ($old_hashes)");
+
 		}
 
-		q("DELETE FROM outq WHERE outq_created < %s - INTERVAL %s",
-			db_utcnow(), db_quoteinterval('3 DAY')
-		);
+		$deliveries = [];
 
 		if ($queue_id) {
-			$r = q("SELECT * FROM outq WHERE outq_hash = '%s' LIMIT 1",
+			$qItems = q("SELECT * FROM outq WHERE outq_hash = '%s' LIMIT 1",
 				dbesc($queue_id)
 			);
+			logger('queue deliver: ' . $qItems[0]['outq_hash'] . ' to ' . $qItems[0]['outq_posturl'], LOGGER_DEBUG);
+			LibQueue::deliver($qItems[0]);
 		}
 		else {
-
-			// For the first 12 hours we'll try to deliver every 15 minutes
-			// After that, we'll only attempt delivery once per hour.
-			// This currently only handles the default queue drivers ('zot' or '') which we will group by posturl
-			// so that we don't start off a thousand deliveries for a couple of dead hubs.
-			// The zot driver will deliver everything destined for a single hub once contact is made (*if* contact is made).
-			// Other drivers will have to do something different here and may need their own query.
-
-			// Note: this requires some tweaking as new posts to long dead hubs once a day will keep them in the
-			// "every 15 minutes" category. We probably need to prioritise them when inserted into the queue
-			// or just prior to this query based on recent and long-term delivery history. If we have good reason to believe
-			// the site is permanently down, there's no reason to attempt delivery at all, or at most not more than once
-			// or twice a day.
-
-			$sqlrandfunc = db_getfunc('rand');
-
-			$r = q("SELECT *,$sqlrandfunc as rn FROM outq WHERE outq_delivered = 0 and outq_scheduled < %s order by rn limit 1",
+			$qItems = q("SELECT outq_hash FROM outq WHERE outq_scheduled < %s ",
 				db_utcnow()
 			);
-			while ($r) {
-				foreach ($r as $rv) {
-					LibQueue::deliver($rv);
-				}
-				$r = q("SELECT *,$sqlrandfunc as rn FROM outq WHERE outq_delivered = 0 and outq_scheduled < %s order by rn limit 1",
-					db_utcnow()
-				);
-			}
-		}
-		if (!$r)
-			return;
 
-		foreach ($r as $rv) {
-			LibQueue::deliver($rv);
-		}
+			if ($qItems) {
+				foreach ($qItems as $qItem) {
+					$deliveries[] = $qItem['outq_hash'];
+				}
+
+				shuffle($deliveries);
+				do_delivery($deliveries, true);
+			}
+		 }
 	}
+
 }

Zotlabs/Daemon/Thumbnail.php

@@ -73,5 +73,7 @@ class Thumbnail {
 				|| (filectime(dbunescbin($attach['content']) . 'thumb') < (time() - 60)))) {
 			$default_controller->Thumb($attach, $preview_style, $preview_width, $preview_height);
 		}
+
+		return;
 	}
 }

Zotlabs/Daemon/Xchan_photo.php

@@ -0,0 +1,34 @@
+<?php
+
+/** @file */
+
+namespace Zotlabs\Daemon;
+
+class Xchan_photo {
+
+	static public function run($argc, $argv) {
+
+		if ($argc != 3) {
+			return;
+		}
+
+		$url   = hex2bin($argv[1]);
+		$xchan = hex2bin($argv[2]);
+
+		$photos = import_xchan_photo($url, $xchan);
+		if ($photos) {
+			$result = q("update xchan set xchan_photo_date = '%s', xchan_photo_l = '%s', xchan_photo_m = '%s', xchan_photo_s = '%s', xchan_photo_mimetype = '%s' where xchan_hash = '%s'",
+				dbescdate(datetime_convert()),
+				dbesc($photos[0]),
+				dbesc($photos[1]),
+				dbesc($photos[2]),
+				dbesc($photos[3]),
+				dbesc($xchan)
+			);
+
+			if (! $result) {
+				logger("xchan update failed for $url");
+			}
+		}
+	}
+}

Zotlabs/Daemon/Zotconvo.php

@@ -0,0 +1,32 @@
+<?php
+
+namespace Zotlabs\Daemon;
+
+use Zotlabs\Lib\Libzot;
+
+class Zotconvo {
+
+	static public function run($argc, $argv) {
+
+		logger('Zotconvo invoked: ' . print_r($argv, true));
+
+		if ($argc != 3) {
+			return;
+		}
+
+		$mid = $argv[2];
+		if (!$mid) {
+			return;
+		}
+
+		$channel = channelx_by_n(intval($argv[1]));
+		if (!$channel) {
+			return;
+		}
+
+		Libzot::fetch_conversation($channel, $mid);
+
+		return;
+
+	}
+}

Zotlabs/Lib/ASCollection.php

@@ -24,6 +24,8 @@ class ASCollection {
 		$this->direction = $direction;
 		$this->limit     = $limit;
 
+		$data = null;
+
 		if (is_array($obj)) {
 			$data = $obj;
 		}
@@ -147,4 +149,4 @@ class ASCollection {
 		}
 		logger('nextpage: ' . $this->nextpage, LOGGER_DEBUG);
 	}
-}
+}

Zotlabs/Lib/AbConfig.php

@@ -6,12 +6,17 @@ namespace Zotlabs\Lib;
 class AbConfig {
 
 	static public function Load($chan,$xhash,$family = '') {
-		if($family)
+		$where = '';
+
+		if($family) {
 			$where = sprintf(" and cat = '%s' ",dbesc($family));
+		}
+
 		$r = q("select * from abconfig where chan = %d and xchan = '%s' $where",
 			intval($chan),
 			dbesc($xhash)
 		);
+
 		return $r;
 	}
 
@@ -21,7 +26,7 @@ class AbConfig {
 			intval($chan),
 			dbesc($xhash),
 			dbesc($family),
-			dbesc($key)		
+			dbesc($key)
 		);
 		if($r) {
 			return ((preg_match('|^a:[0-9]+:{.*}$|s', $r[0]['v'])) ? unserialize($r[0]['v']) : $r[0]['v']);
@@ -41,19 +46,19 @@ class AbConfig {
 				dbesc($xhash),
 				dbesc($family),
 				dbesc($key),
-				dbesc($dbvalue)		
+				dbesc($dbvalue)
 			);
 		}
 		else {
 			$r = q("update abconfig set v = '%s' where chan = %d and xchan = '%s' and cat = '%s' and k = '%s' ",
-				dbesc($dbvalue),		
+				dbesc($dbvalue),
 				dbesc($chan),
 				dbesc($xhash),
 				dbesc($family),
 				dbesc($key)
 			);
 		}
-	
+
 		if($r)
 			return $value;
 		return false;

Zotlabs/Lib/AccessList.php

@@ -311,55 +311,6 @@ class AccessList {
 		return $o;
 	}
 
-/* deprecated
-	static function widget($every = "connections", $each = "lists", $edit = false, $group_id = 0, $cid = '', $mode = 1) {
-
-		$groups = [];
-
-		$r         = q("SELECT * FROM pgrp WHERE deleted = 0 AND uid = %d ORDER BY gname ASC",
-			intval($_SESSION['uid'])
-		);
-		$member_of = [];
-		if ($cid) {
-			$member_of = self::containing(local_channel(), $cid);
-		}
-
-		if ($r) {
-			foreach ($r as $rr) {
-				$selected = (($group_id == $rr['id']) ? ' group-selected' : '');
-
-				if ($edit) {
-					$groupedit = ['href' => "lists/" . $rr['id'], 'title' => t('edit')];
-				}
-				else {
-					$groupedit = null;
-				}
-
-				$groups[] = [
-					'id'       => $rr['id'],
-					'enc_cid'  => base64url_encode($cid),
-					'cid'      => $cid,
-					'text'     => $rr['gname'],
-					'selected' => $selected,
-					'href'     => (($mode == 0) ? $each . '?f=&gid=' . $rr['id'] : $each . "/" . $rr['id']) . ((x($_GET, 'new')) ? '&new=' . $_GET['new'] : '') . ((x($_GET, 'order')) ? '&order=' . $_GET['order'] : ''),
-					'edit'     => $groupedit,
-					'ismember' => in_array($rr['id'], $member_of),
-				];
-			}
-		}
-
-		return replace_macros(get_markup_template('group_side.tpl'), [
-			'$title'      => t('Privacy Groups'),
-			'$edittext'   => t('Edit group'),
-			'$createtext' => t('Create new group'),
-			'$ungrouped'  => (($every === 'contacts') ? t('Channels not in any privacy group') : ''),
-			'$groups'     => $groups,
-			'$add'        => t('Add'),
-		]);
-
-	}
-*/
-
 	static function expand($g) {
 		if (!(is_array($g) && count($g))) {
 			return [];

Zotlabs/Lib/ActivityStreams.php

@@ -14,6 +14,7 @@ class ActivityStreams {
 	public $meta = null;
 	public $valid = false;
 	public $deleted = false;
+	public $portable_id = null;
 	public $id = '';
 	public $parent_id = '';
 	public $type = '';
@@ -35,12 +36,13 @@ class ActivityStreams {
 	 *
 	 * @param string $string
 	 */
-	function __construct($string) {
+	function __construct($string, $portable_id = null) {
 
 		if(!$string)
 			return;
 
 		$this->raw = $string;
+		$this->portable_id = $portable_id;
 
 		if (is_array($string)) {
 			$this->data = $string;
@@ -70,9 +72,12 @@ class ActivityStreams {
 				}
 			}
 
+			// This indicates only that we have sucessfully decoded JSON.
 			$this->valid = true;
 
-			if (array_key_exists('type', $this->data) && array_key_exists('actor', $this->data) && array_key_exists('object', $this->data)) {
+			// Special handling for Mastodon "delete actor" activities which will often fail to verify
+			// because the key cannot be fetched. We will catch this condition elsewhere.
+			if (is_array($this->data) && array_key_exists('type', $this->data) && array_key_exists('actor', $this->data) && array_key_exists('object', $this->data)) {
 				if ($this->data['type'] === 'Delete' && $this->data['actor'] === $this->data['object']) {
 					$this->deleted = $this->data['actor'];
 					$this->valid   = false;
@@ -81,6 +86,7 @@ class ActivityStreams {
 
 		}
 
+		// Attempt to assemble an Activity from what we were given.
 		if ($this->is_valid()) {
 			$this->id     = $this->get_property_obj('id');
 			$this->type   = $this->get_primary_type();
@@ -109,22 +115,24 @@ class ActivityStreams {
 			// fetch recursive or embedded activities
 
 			if ($this->obj && is_array($this->obj) && array_key_exists('object', $this->obj)) {
-				$this->obj['object'] = $this->get_compound_property($this->obj['object']);
+				$this->obj['object'] = $this->get_compound_property('object', $this->obj);
 			}
 
-			if ($this->obj && is_array($this->obj) && $this->obj['actor'])
+			if ($this->obj && is_array($this->obj) && isset($this->obj['actor']))
 				$this->obj['actor'] = $this->get_actor('actor', $this->obj);
-			if ($this->tgt && is_array($this->tgt) && $this->tgt['actor'])
+			if ($this->tgt && is_array($this->tgt) && isset($this->tgt['actor']))
 				$this->tgt['actor'] = $this->get_actor('actor', $this->tgt);
 
 			$this->parent_id = $this->get_property_obj('inReplyTo');
 
-			if ((!$this->parent_id) && is_array($this->obj)) {
+			if (!$this->parent_id && is_array($this->obj) && isset($this->obj['inReplyTo'])) {
 				$this->parent_id = $this->obj['inReplyTo'];
 			}
-			if ((!$this->parent_id) && is_array($this->obj)) {
+
+			if (!$this->parent_id && is_array($this->obj) && isset($this->obj['id'])) {
 				$this->parent_id = $this->obj['id'];
 			}
+
 		}
 	}
 
@@ -141,36 +149,59 @@ class ActivityStreams {
 		$this->saved_recips = $arr;
 	}
 
+	/**
+	 * @brief get single property from Activity object
+	 *
+	 * @param string $property
+	 * @param mixed $default return value if property or object not set
+	 *    or object is a string id which could not be fetched.
+	 * @return mixed
+	 */
+	public function objprop(string $property, mixed $default = false): mixed {
+		$x = $this->get_property_obj($property, $this->obj);
+		return (isset($x)) ? $x : $default;
+	}
+
 	/**
 	 * @brief Collects all recipients.
 	 *
-	 * @param string $base
+	 * @param mixed $base
 	 * @param string $namespace (optional) default empty
 	 * @return array
 	 */
-	function collect_recips($base = '', $namespace = '') {
-		$x = [];
+	public function collect_recips(mixed $base = '', string $namespace = ''): array {
+		$result = [];
+		$tmp = [];
 
 		$fields = ['to', 'cc', 'bto', 'bcc', 'audience'];
-		foreach ($fields as $f) {
-			$y = $this->get_compound_property($f, $base, $namespace);
-			if ($y) {
-				if (!is_array($this->raw_recips)) {
-					$this->raw_recips = [];
-				}
-
-				if (!is_array($y)) {
-					$y = [$y];
-				}
-				$this->raw_recips[$f] = $y;
-				$x                    = array_merge($x, $y);
+		foreach ($fields as $field) {
+			// don't expand these yet
+			$values = $this->get_property_obj($field, $base, $namespace);
+			if ($values) {
+				$values = force_array($values);
+				$tmp[$field] = $values;
+				$result = array_values(array_unique(array_merge($result, $values)));
+			}
+			// Merge the object recipients if they exist.
+			$values = $this->objprop($field);
+			if ($values) {
+				$values = force_array($values);
+				$tmp[$field] = ((isset($tmp[$field])) ? array_merge($tmp[$field], $values) : $values);
+				$result = array_values(array_unique(array_merge($result, $values)));
+			}
+			// remove duplicates
+			if (isset($tmp[$field])) {
+				$tmp[$field] = array_values(array_unique($tmp[$field]));
 			}
 		}
-// not yet ready for prime time
-//		$x = $this->expand($x,$base,$namespace);
-		return $x;
+		$this->raw_recips = $tmp;
+
+		// not yet ready for prime time
+		//      $result = $this->expand($result,$base,$namespace);
+		return $result;
 	}
 
+
 	function expand($arr, $base = '', $namespace = '') {
 		$ret = [];
 
@@ -274,12 +305,26 @@ class ActivityStreams {
 	 * @return NULL|mixed
 	 */
 
-	function fetch_property($url) {
-		return self::fetch($url);
-	}
+	function fetch_property($url, $channel = null) {
+		$x = null;
 
-	static function fetch($url, $channel = null) {
-		return Activity::fetch($url, $channel);
+		if (str_starts_with($url, z_root() . '/item/')) {
+			$x = Activity::fetch_local($url, $this->portable_id ?? '');
+			logger('local: ' . print_r($x,true));
+		}
+
+		if (!$x) {
+			$x = Activity::fetch($url, $channel);
+			if ($x === null && strpos($url, '/channel/')) {
+				// look for other nomadic channels which might be alive
+				$zf = Zotfinger::exec($url, $channel);
+
+				$url = $zf['signature']['signer'];
+				$x = Activity::fetch($url, $channel);
+			}
+		}
+
+		return $x;
 	}
 
 	static function is_an_actor($s) {
@@ -290,7 +335,7 @@ class ActivityStreams {
 		if (!$s) {
 			return false;
 		}
-		return (in_array($s, ['Like', 'Dislike', 'Flag', 'Block', 'Announce', 'Accept', 'Reject', 'TentativeAccept', 'TentativeReject', 'emojiReaction', 'EmojiReaction', 'EmojiReact']));
+		return (in_array($s, ['Like', 'Dislike', 'Flag', 'Block', 'Accept', 'Reject', 'TentativeAccept', 'TentativeReject', 'emojiReaction', 'EmojiReaction', 'EmojiReact']));
 	}
 
 	/**
@@ -304,20 +349,25 @@ class ActivityStreams {
 
 	function get_actor($property, $base = '', $namespace = '') {
 		$x = $this->get_property_obj($property, $base, $namespace);
+
 		if ($this->is_url($x)) {
-			$y = Activity::get_cached_actor($x);
+			$y = Activity::get_actor($x);
 			if ($y) {
 				return $y;
 			}
 		}
+
 		$actor = $this->get_compound_property($property, $base, $namespace, true);
+
 		if (is_array($actor) && self::is_an_actor($actor['type'])) {
 			if (array_key_exists('id', $actor) && (!array_key_exists('inbox', $actor))) {
 				$actor = $this->fetch_property($actor['id']);
 			}
 			return $actor;
 		}
-		return null;
+
+		return Activity::get_unknown_actor($this->data);
+
 	}
 
 
@@ -332,6 +382,7 @@ class ActivityStreams {
 	 */
 	function get_compound_property($property, $base = '', $namespace = '', $first = false) {
 		$x = $this->get_property_obj($property, $base, $namespace);
+
 		if ($this->is_url($x)) {
 			$y = $this->fetch_property($x);
 			if (is_array($y)) {
@@ -422,15 +473,19 @@ class ActivityStreams {
 
 	static function get_accept_header_string($channel = null) {
 
+		$ret = '';
+
 		$hookdata = [];
 		if ($channel)
 			$hookdata['channel'] = $channel;
 
-		$hookdata['data'] = 'application/x-zot-activity+json';
+		$hookdata['data'] = ['application/x-zot-activity+json'];
 
 		call_hooks('get_accept_header_string', $hookdata);
 
-		return $hookdata['data'];
+		$ret = implode(', ', $hookdata['data']);
+
+		return $ret;
 
 	}
 

Zotlabs/Lib/Apps.php

@@ -159,7 +159,7 @@ class Apps {
 		foreach(self::$available_apps as $iapp) {
 			if($iapp['app_id'] == hash('whirlpool',$app['name'])) {
 				$notfound = false;
-				if(($iapp['app_version'] !== $app['version'])
+				if((isset($app['version']) && $iapp['app_version'] !== $app['version'])
 					|| ($app['plugin'] && (! $iapp['app_plugin']))) {
 					return intval($iapp['app_id']);
 				}
@@ -236,6 +236,7 @@ class Apps {
 			$ret['photo'] = $baseurl . '/' . get_default_profile_photo(80);
 
 		$ret['type'] = 'system';
+		$ret['plugin'] = '';
 
 		foreach($ret as $k => $v) {
 			if(strpos($v,'http') === 0) {
@@ -418,11 +419,28 @@ class Apps {
 	static public function app_render($papp, $mode = 'view') {
 		$installed = false;
 
-		if(! $papp)
+		if(!$papp) {
 			return;
+		}
 
-		if(! $papp['photo'])
+		/**
+		 * @hooks app_render_before
+		 * Hook to manipulate the papp array before rendering
+		 */
+
+		$hookinfo = [
+			'name' => $papp['name'],
+			'photo' => $papp['photo']
+		];
+
+		call_hooks('app_render_manipulate_photo', $hookinfo);
+
+		// We will only allow to manipulate the photo
+		$papp['photo'] = $hookinfo['photo'];
+
+		if(!$papp['photo']) {
 			$papp['photo'] = 'icon:gear';
+		}
 
 		self::translate_system_apps($papp);
 
@@ -521,8 +539,13 @@ class Apps {
 		$hosturl = '';
 
 		if(local_channel()) {
-			if(self::app_installed(local_channel(),$papp) && !$papp['deleted'])
+			if(self::app_installed(local_channel(),$papp)) {
 				$installed = true;
+			}
+
+			if ($installed && isset($papp['deleted']) && $papp['deleted']) {
+				$installed = false;
+			}
 
 			$hosturl = z_root() . '/';
 		}
@@ -595,12 +618,12 @@ class Apps {
 			'$edit' => ((local_channel() && $installed && $mode == 'edit') ? t('Edit') : ''),
 			'$delete' => ((local_channel() && $mode == 'edit') ? t('Delete') : ''),
 			'$undelete' => ((local_channel() && $mode == 'edit') ? t('Undelete') : ''),
-			'$settings_url' => ((local_channel() && $installed && $mode == 'list') ? $papp['settings_url'] : ''),
-			'$deleted' => $papp['deleted'],
+			'$settings_url' => ((local_channel() && $installed && $mode == 'list' && isset($papp['settings_url'])) ? $papp['settings_url'] : ''),
+			'$deleted' => $papp['deleted'] ?? false,
 			'$feature' => ((isset($papp['embed']) || $mode == 'edit') ? false : true),
 			'$pin' => ((isset($papp['embed']) || $mode == 'edit') ? false : true),
-			'$featured' => ((strpos($papp['categories'], 'nav_featured_app') === false) ? false : true),
-			'$pinned' => ((strpos($papp['categories'], 'nav_pinned_app') === false) ? false : true),
+			'$featured' => ((isset($papp['categories']) && strpos($papp['categories'], 'nav_featured_app') !== false) ? true : false),
+			'$pinned' => ((isset($papp['categories']) && strpos($papp['categories'], 'nav_pinned_app') !== false) ? true : false),
 			'$mode' => $mode,
 			'$add' => t('Add to app-tray'),
 			'$remove' => t('Remove from app-tray'),
@@ -639,7 +662,7 @@ class Apps {
 			);
 			if($r) {
 				if($app['uid']) {
-					if($app['categories'] && (! $app['term'])) {
+					if((isset($app['categories']) && $app['categories']) && !(isset($app['term']) && $app['term'])) {
 						$r[0]['term'] = q("select * from term where otype = %d and oid = %d",
 							intval(TERM_OBJ_APP),
 							intval($r[0]['id'])
@@ -1184,7 +1207,7 @@ class Apps {
 			$ret['success'] = true;
 			$ret['app_id'] = $darray['app_id'];
 		}
-		if($arr['categories']) {
+		if(isset($arr['categories']) && $arr['categories']) {
 			$x = q("select id from app where app_id = '%s' and app_channel = %d limit 1",
 				dbesc($darray['app_id']),
 				intval($darray['app_channel'])
@@ -1282,7 +1305,7 @@ class Apps {
 				intval(TERM_OBJ_APP),
 				intval($x[0]['id'])
 			);
-			if($arr['categories']) {
+			if(isset($arr['categories']) && $arr['categories']) {
 				$y = explode(',',$arr['categories']);
 				if($y) {
 					foreach($y as $t) {

Zotlabs/Lib/Cache.php

@@ -5,17 +5,17 @@ namespace Zotlabs\Lib;
 	/**
 	 *  cache api
 	 */
-	 
+
 class Cache {
-    
+
     /**
      * @brief Returns cached content
-     * 
+     *
      * @param string $key
      * @param string $age in SQL format, default is '30 DAY'
      * @return string
      */
-    
+
 	public static function get($key, $age = '') {
 
 		$hash = hash('whirlpool',$key);
@@ -25,12 +25,12 @@ class Cache {
 			db_utcnow(),
 			db_quoteinterval(($age ? $age : get_config('system','object_cache_days', '30') . ' DAY'))
 		);
-			
+
 		if ($r)
 			return $r[0]['v'];
 		return null;
 	}
-		
+
 	public static function set($key,$value) {
 
 		$hash = hash('whirlpool',$key);
@@ -45,7 +45,7 @@ class Cache {
 				dbesc($hash));
 		}
 		else {
-			q("INSERT INTO cache ( k, v, updated) VALUES ('%s','%s','%s')",
+			q("INSERT INTO cache (k, v, updated) VALUES ('%s', '%s', '%s')",
 				dbesc($hash),
 				dbesc($value),
 				dbesc(datetime_convert()));

Zotlabs/Lib/Connect.php

@@ -69,7 +69,8 @@ class Connect {
 		$xchan_hash = '';
 		$sql_options = (($protocol) ? " and xchan_network = '" . dbesc($protocol) . "' " : '');
 
-		$r = q("select * from xchan where ( xchan_hash = '%s' or xchan_url = '%s' or xchan_addr = '%s') $sql_options ",
+		// We need both, the xchan and the hubloc here hence use JOIN instead of LEFT JOIN
+		$r = q("SELECT * FROM xchan JOIN hubloc ON xchan_hash = hubloc_hash WHERE ( xchan_hash = '%s' or xchan_url = '%s' or xchan_addr = '%s') $sql_options ORDER BY hubloc_id DESC",
 			dbesc($url),
 			dbesc($url),
 			dbesc($url)
@@ -80,12 +81,13 @@ class Connect {
 			// reset results to the best record or the first if we don't have the best
 			// note: this is a single record and not an array of results
 
-			$r = Libzot::zot_record_preferred($r,'xchan_network');
+			$r = Libzot::zot_record_preferred($r, 'xchan_network');
 
 		}
 
 		$singleton = false;
 		$d = false;
+		$wf = false;
 
 		if (! $r) {
 
@@ -108,10 +110,12 @@ class Connect {
 
 		if ($wf || $d) {
 
+			$xchan_hash = (($wf) ? $wf : $url);
+
 			// something was discovered - find the record which was just created.
 
 			$r = q("select * from xchan where ( xchan_hash = '%s' or xchan_url = '%s' or xchan_addr = '%s' ) $sql_options",
-				dbesc(($wf) ? $wf : $url),
+				dbesc($xchan_hash),
 				dbesc($url),
 				dbesc($url)
 			);
@@ -119,7 +123,7 @@ class Connect {
 			// convert to a single record (once again preferring a zot solution in the case of multiples)
 
 			if ($r) {
-				$r = Libzot::zot_record_preferred($r,'xchan_network');
+				$r = Libzot::zot_record_preferred($r, 'xchan_network');
 			}
 		}
 

Zotlabs/Lib/Crypto.php

@@ -87,6 +87,10 @@ class Crypto {
 			return false;
 		}
 
+		if (!$alg) {
+			$alg = 'sha256';
+		}
+
 		try {
 			$verify = openssl_verify($data, $sig, $key, $alg);
 		} catch (Exception $e) {

Zotlabs/Lib/DReport.php

@@ -94,19 +94,6 @@ class DReport {
 		if(! $c)
 			return false;
 
-		// legacy zot recipients add a space and their name to the xchan. remove it if true.
-
-		$legacy_recipient = strpos($dr['recipient'], ' ');
-		if($legacy_recipient !== false) {
-			$legacy_recipient_parts = explode(' ', $dr['recipient'], 2);
-			$rxchan = $legacy_recipient_parts[0];
-		}
-		else {
-			$rxchan = $dr['recipient'];
-		}
-
-
-
 		// is the recipient one of our connections, or do we want to store every report?
 
 		$pcf = get_pconfig($c[0]['channel_id'],'system','dreport_store_all');
@@ -117,7 +104,7 @@ class DReport {
 		// So if a remote site says they can't find us, that's no big surprise
 		// and just creates a lot of extra report noise
 
-		if(($dr['location'] !== z_root()) && ($dr['sender'] === $rxchan) && ($dr['status'] === 'recipient not found'))
+		if(($dr['location'] !== z_root()) && ($dr['sender'] === $dr['recipient']) && ($dr['status'] === 'recipient not found'))
 			return false;
 
 		// If you have a private post with a recipient list, every single site is going to report
@@ -126,14 +113,14 @@ class DReport {
 		// have a channel on that site.
 
 		$r = q("select hubloc_id from hubloc where hubloc_hash = '%s' and hubloc_url = '%s'",
-			dbesc($rxchan),
+			dbesc($dr['recipient']),
 			dbesc($dr['location'])
 		);
 		if((! $r) && ($dr['status'] === 'recipient_not_found'))
 			return false;
 
 		$r = q("select abook_id from abook where abook_xchan = '%s' and abook_channel = %d limit 1",
-			dbesc($rxchan),
+			dbesc($dr['recipient']),
 			intval($c[0]['channel_id'])
 		);
 		if($r)

Zotlabs/Lib/Enotify.php

@@ -122,8 +122,11 @@ class Enotify {
 	// e.g. "your post", "David's photo", etc.
 	$possess_desc = t('%s <!item_type!>');
 
+	$parent_mid = '';
+	$parent_item = [];
+
 // @@TODO: consider using switch instead of those elseif
-	if ($params['type'] == NOTIFY_MAIL) {
+	if (isset($params['type']) && $params['type'] == NOTIFY_MAIL) {
 		logger('notification: mail');
 		$subject = 	sprintf( t('[$Projectname:Notify] New direct message received at %s'), $sitename);
 
@@ -135,14 +138,14 @@ class Enotify {
 		$itemlink = $siteurl . '/hq/' . gen_link_id($params['item']['mid']);
 	}
 
-	elseif ($params['type'] === NOTIFY_COMMENT) {
+	elseif (isset($params['type']) && $params['type'] === NOTIFY_COMMENT) {
 		//logger("notification: params = " . print_r($params, true), LOGGER_DEBUG);
 
 		$moderated = (($params['item']['item_blocked'] == ITEM_MODERATED) ? true : false);
 
 		$itemlink = $params['link'];
 
-		$action = t('commented on');
+		$action = (($moderated) ? t('requested to comment on') : t('commented on'));
 
 		if(array_key_exists('item',$params)) {
 
@@ -155,10 +158,10 @@ class Enotify {
 				}
 
 				if(activity_match($params['verb'], ACTIVITY_LIKE))
-					$action = t('liked');
+					$action = (($moderated) ? t('requested to like') : t('liked'));
 
 				if(activity_match($params['verb'], ACTIVITY_DISLIKE))
-					$action = t('disliked');
+					$action = (($moderated) ? t('requested to dislike') : t('disliked'));
 
 			}
 
@@ -167,7 +170,7 @@ class Enotify {
 
 		}
 
-		$parent_mid = $params['parent_mid'];
+		$parent_mid = $params['parent_mid'] ?? '';
 
 		// Check to see if there was already a notify for this post.
 		// If so don't create a second notification
@@ -251,20 +254,20 @@ class Enotify {
 
 	}
 
-	elseif ($params['type'] === NOTIFY_LIKE) {
+	elseif (isset($params['type']) && $params['type'] === NOTIFY_LIKE) {
 //		logger("notification: params = " . print_r($params, true), LOGGER_DEBUG);
 
 		$itemlink =  $params['link'];
 
-		if (array_key_exists('item',$params) && activity_match($params['item']['verb'],ACTIVITY_LIKE)) {
-			if(! $always_show_in_notices  || !($vnotify & VNOTIFY_LIKE)) {
+		if (array_key_exists('item',$params) && (activity_match($params['item']['verb'], ACTIVITY_LIKE) || activity_match($params['item']['verb'], ACTIVITY_DISLIKE))) {
+			if(! $always_show_in_notices  || !($vnotify & VNOTIFY_LIKE) || !feature_enabled($recip['channel_id'], 'dislike')) {
 				logger('notification: not a visible activity. Ignoring.');
 				pop_lang();
 				return;
 			}
 		}
 
-		$parent_mid = $params['parent_mid'];
+		$parent_mid = $params['parent_mid'] ?? '';
 
 		// Check to see if there was already a notify for this post.
 		// If so don't create a second notification
@@ -291,22 +294,36 @@ class Enotify {
 			);
 		}
 
+		if (!$p) {
+			pop_lang();
+			return;
+		}
+
 		xchan_query($p);
 
-//@@FIXME $p can be null (line 285)
 		$item_post_type = item_post_type($p[0]);
 //		$private = $p[0]['item_private'];
 		$parent_id = $p[0]['id'];
 
 		$parent_item = $p[0];
 
+		//$verb = ((activity_match($params['item']['verb'], ACTIVITY_DISLIKE)) ? t('disliked') : t('liked'));
+		$moderated = (($params['item']['item_blocked'] == ITEM_MODERATED) ? true : false);
+
+		if(activity_match($params['item']['verb'], ACTIVITY_LIKE))
+			$verb = (($moderated) ? t('requested to like') : t('liked'));
+
+		if(activity_match($params['item']['verb'], ACTIVITY_DISLIKE))
+			$verb = (($moderated) ? t('requested to dislike') : t('disliked'));
 
 		// "your post"
 		if($p[0]['owner']['xchan_name'] === $p[0]['author']['xchan_name'] && intval($p[0]['item_wall']))
-			$dest_str = sprintf(t('%1$s liked [zrl=%2$s]your %3$s[/zrl]'),
+			$dest_str = sprintf(t('%1$s %2$s [zrl=%3$s]your %4$s[/zrl]'),
 				'[zrl=' . $sender['xchan_url'] . ']' . $sender['xchan_name'] . '[/zrl]',
+				$verb,
 				$itemlink,
-				$item_post_type);
+				$item_post_type
+			);
 		else {
 			pop_lang();
 			return;
@@ -328,7 +345,7 @@ class Enotify {
 
 
 
-	elseif($params['type'] === NOTIFY_WALL) {
+	elseif(isset($params['type']) && $params['type'] === NOTIFY_WALL) {
 		$subject = sprintf( t('[$Projectname:Notify] %s posted to your profile wall') , $sender['xchan_name']);
 
 		$preamble = sprintf( t('%1$s posted to your profile wall at %2$s') , $sender['xchan_name'], $sitename);
@@ -343,7 +360,7 @@ class Enotify {
 		$itemlink =  $params['link'];
 	}
 
-	elseif ($params['type'] === NOTIFY_TAGSELF) {
+	elseif (isset($params['type']) && $params['type'] === NOTIFY_TAGSELF) {
 
 		$p = q("select id from notify where link = '%s' and uid = %d limit 1",
 			dbesc($params['link']),
@@ -367,7 +384,7 @@ class Enotify {
 		$itemlink =  $params['link'];
 	}
 
-	elseif ($params['type'] === NOTIFY_POKE) {
+	elseif (isset($params['type']) && $params['type'] === NOTIFY_POKE) {
 		$subject =	sprintf( t('[$Projectname:Notify] %1$s poked you') , $sender['xchan_name']);
 		$preamble = sprintf( t('%1$s poked you at %2$s') , $sender['xchan_name'], $sitename);
 		$epreamble = sprintf( t('%1$s [zrl=%2$s]poked you[/zrl].') ,
@@ -384,7 +401,7 @@ class Enotify {
 		$itemlink =  $params['link'];
 	}
 
-	elseif ($params['type'] === NOTIFY_TAGSHARE) {
+	elseif (isset($params['type']) && $params['type'] === NOTIFY_TAGSHARE) {
 		$subject =	sprintf( t('[$Projectname:Notify] %s tagged your post') , $sender['xchan_name']);
 		$preamble = sprintf( t('%1$s tagged your post at %2$s'),$sender['xchan_name'], $sitename);
 		$epreamble = sprintf( t('%1$s tagged [zrl=%2$s]your post[/zrl]') ,
@@ -397,7 +414,7 @@ class Enotify {
 		$itemlink =  $params['link'];
 	}
 
-	elseif ($params['type'] === NOTIFY_INTRO) {
+	elseif (isset($params['type']) && $params['type'] === NOTIFY_INTRO) {
 		$subject = sprintf( t('[$Projectname:Notify] Introduction received'));
 		$preamble = sprintf( t('You\'ve received an new connection request from \'%1$s\' at %2$s'), $sender['xchan_name'], $sitename);
 		$epreamble = sprintf( t('You\'ve received [zrl=%1$s]a new connection request[/zrl] from %2$s.'),
@@ -411,7 +428,7 @@ class Enotify {
 		$itemlink = $params['link'];
 	}
 
-	elseif ($params['type'] === NOTIFY_SUGGEST) {
+	elseif (isset($params['type']) && $params['type'] === NOTIFY_SUGGEST) {
 		$subject = sprintf( t('[$Projectname:Notify] Friend suggestion received'));
 		$preamble = sprintf( t('You\'ve received a friend suggestion from \'%1$s\' at %2$s'), $sender['xchan_name'], $sitename);
 		$epreamble = sprintf( t('You\'ve received [zrl=%1$s]a friend suggestion[/zrl] for %2$s from %3$s.'),
@@ -429,11 +446,11 @@ class Enotify {
 		$itemlink =  $params['link'];
 	}
 
-	elseif ($params['type'] === NOTIFY_CONFIRM) {
+	elseif (isset($params['type']) && $params['type'] === NOTIFY_CONFIRM) {
 		// ?
 	}
 
-	elseif ($params['type'] === NOTIFY_SYSTEM) {
+	elseif (isset($params['type']) && $params['type'] === NOTIFY_SYSTEM) {
 		// ?
 	}
 
@@ -488,12 +505,13 @@ class Enotify {
 	$datarray['link']   = $itemlink;
 	$datarray['parent'] = $parent_mid;
 	$datarray['parent_item'] = $parent_item;
-	$datarray['ntype']   = $params['type'];
-	$datarray['verb']   = $params['verb'];
-	$datarray['otype']  = $params['otype'];
+	$datarray['ntype']   = $params['type'] ?? '';
+	$datarray['verb']   = $params['verb'] ?? '';
+	$datarray['otype']  = $params['otype'] ?? '';
  	$datarray['abort']  = false;
+	$datarray['seen'] = 0;
 
-	$datarray['item'] = $params['item'];
+	$datarray['item'] = $params['item'] ?? [];
 
 	call_hooks('enotify_store', $datarray);
 
@@ -504,7 +522,6 @@ class Enotify {
 
 
 	// create notification entry in DB
-	$seen = 0;
 
 	// Mark some notifications as seen right away
 	// Note! The notification have to be created, because they are used to send emails
@@ -514,7 +531,7 @@ class Enotify {
 
 	if (!$always_show_in_notices) {
 		if (($params['type'] === NOTIFY_WALL) || ($params['type'] === NOTIFY_MAIL) || ($params['type'] === NOTIFY_INTRO)) {
-			$seen = 1;
+			$datarray['seen'] = 1;
 		}
 	}
 
@@ -530,7 +547,7 @@ class Enotify {
 		intval($datarray['uid']),
 		dbesc($datarray['link']),
 		dbesc($datarray['parent']),
-		intval($seen),
+		intval($datarray['seen']),
 		intval($datarray['ntype']),
 		dbesc($datarray['verb']),
 		dbesc($datarray['otype'])
@@ -606,8 +623,8 @@ class Enotify {
 		$datarray['preamble']     = $preamble;
 		$datarray['sitename']     = $sitename;
 		$datarray['siteurl']      = $siteurl;
-		$datarray['type']         = $params['type'];
-		$datarray['parent']       = $params['parent_mid'];
+		$datarray['type']         = $params['type'] ?? '';
+		$datarray['parent']       = $params['parent_mid'] ?? '';
 		$datarray['source_name']  = $sender['xchan_name'];
 		$datarray['source_link']  = $sender['xchan_url'];
 		$datarray['source_photo'] = $sender['xchan_photo_s'];
@@ -674,7 +691,6 @@ class Enotify {
 			'$source_name'  => $datarray['source_name'],
 			'$source_link'  => $datarray['source_link'],
 			'$source_photo' => $datarray['source_photo'],
-			'$username'     => $datarray['to_name'],
 			'$hsitelink'    => $datarray['hsitelink'],
 			'$hitemlink'    => $datarray['hitemlink'],
 			'$thanks'       => $datarray['thanks'],
@@ -696,7 +712,6 @@ class Enotify {
 			'$source_name'  => $datarray['source_name'],
 			'$source_link'  => $datarray['source_link'],
 			'$source_photo' => $datarray['source_photo'],
-			'$username'     => $datarray['to_name'],
 			'$tsitelink'    => $datarray['tsitelink'],
 			'$titemlink'    => $datarray['titemlink'],
 			'$thanks'       => $datarray['thanks'],
@@ -808,10 +823,10 @@ class Enotify {
 
 		localize_item($item);
 
-		if($item['shortlocalize']) {
+		if(isset($item['shortlocalize'])) {
 			$itemem_text = $item['shortlocalize'];
 		}
-		elseif($item['localize']) {
+		elseif(isset($item['localize'])) {
 			$itemem_text = $item['localize'];
 		}
 		else {
@@ -820,10 +835,18 @@ class Enotify {
 				: (($item['obj_type'] === 'Answer') ? sprintf( t('voted on %s\'s poll'), '[bdi]' . $item['owner']['xchan_name'] . '[/bdi]') : sprintf( t('commented on %s\'s post'), '[bdi]' . $item['owner']['xchan_name'] . '[/bdi]'))
 			);
 
-			if($item['verb'] === ACTIVITY_SHARE) {
+			if($item['verb'] === ACTIVITY_SHARE && empty($item['owner']['xchan_pubforum'])) {
 				$itemem_text = sprintf( t('repeated %s\'s post'), '[bdi]' . $item['author']['xchan_name'] . '[/bdi]');
 			}
 
+			if($item['verb'] === ACTIVITY_LIKE) {
+				$itemem_text = sprintf( t('liked %s\'s post'), '[bdi]' . $item['author']['xchan_name'] . '[/bdi]');
+			}
+
+			if($item['verb'] === ACTIVITY_DISLIKE) {
+				$itemem_text = sprintf( t('disliked %s\'s post'), '[bdi]' . $item['author']['xchan_name'] . '[/bdi]');
+			}
+
 			if(in_array($item['obj_type'], ['Document', 'Video', 'Audio', 'Image'])) {
 				$itemem_text = t('shared a file with you');
 			}
@@ -844,12 +867,16 @@ class Enotify {
 
 		// convert this logic into a json array just like the system notifications
 
-		$who = (($item['verb'] === ACTIVITY_SHARE) ? 'owner' : 'author');
+		$who = (($item['verb'] === ACTIVITY_SHARE && empty($item['owner']['xchan_pubforum'])) ? 'owner' : 'author');
+		$body = html2plain(bbcode($item['body'], ['drop_media' => true, 'tryoembed' => false]), 75, true);
+		if ($body) {
+			$body = htmlentities($body, ENT_QUOTES, 'UTF-8', false);
+		}
 
 		$x = array(
 			'notify_link' => $item['llink'],
 			'name' => $item[$who]['xchan_name'],
-			'addr' => (($item[$who]['xchan_addr']) ? $item[$who]['xchan_addr'] : $item[$who]['xchan_url']),
+			'addr' => $item[$who]['xchan_addr'] ? $item[$who]['xchan_addr'] : $item[$who]['xchan_url'],
 			'url' => $item[$who]['xchan_url'],
 			'photo' => $item[$who]['xchan_photo_s'],
 			'when' => (($edit) ? datetime_convert('UTC', date_default_timezone_get(), $item['edited']) : datetime_convert('UTC', date_default_timezone_get(), $item['created'])),
@@ -858,7 +885,7 @@ class Enotify {
 			//'b64mid' => ((in_array($item['verb'], [ACTIVITY_LIKE, ACTIVITY_DISLIKE])) ? gen_link_id($item['thr_parent']) : gen_link_id($item['mid'])),
 			'thread_top' => (($item['item_thread_top']) ? true : false),
 			'message' => bbcode(escape_tags($itemem_text)),
-			'body' =>  htmlentities(html2plain(bbcode($item['body'], ['drop_media', true]), 75, true), ENT_QUOTES, 'UTF-8', false),
+			'body' => $body,
 			// these are for the superblock addon
 			'hash' => $item[$who]['xchan_hash'],
 			'uid' => $item['uid'],

Zotlabs/Lib/IConfig.php

@@ -13,7 +13,7 @@ class IConfig {
 	static public function Get(&$item, $family, $key, $default = false) {
 
 		$is_item = false;
-	
+
 		if(is_array($item)) {
 			$is_item = true;
 			if((! array_key_exists('iconfig',$item)) || (! is_array($item['iconfig'])))
@@ -22,7 +22,7 @@ class IConfig {
 			if(array_key_exists('item_id',$item))
 				$iid = $item['item_id'];
 			else
-				$iid = $item['id'];
+				$iid = $item['id'] ?? 0;
 		}
 		elseif(intval($item))
 			$iid = $item;
@@ -36,7 +36,7 @@ class IConfig {
 					return $c['v'];
 			}
 		}
-		 
+
 		$r = q("select * from iconfig where iid = %d and cat = '%s' and k = '%s' limit 1",
 			intval($iid),
 			dbesc($family),
@@ -63,11 +63,11 @@ class IConfig {
 	 * $value - value of meta variable
 	 * $sharing - boolean (default false); if true the meta information is propagated with the item
 	 *   to other sites/channels, mostly useful when $item is an array and has not yet been stored/delivered.
-	 *   If the meta information is added after delivery and you wish it to be shared, it may be necessary to 
-	 *   alter the item edited timestamp and invoke the delivery process on the updated item. The edited 
+	 *   If the meta information is added after delivery and you wish it to be shared, it may be necessary to
+	 *   alter the item edited timestamp and invoke the delivery process on the updated item. The edited
 	 *   timestamp needs to be altered in order to trigger an item_store_update() at the receiving end.
 	 */
- 
+
 
 	static public function Set(&$item, $family, $key, $value, $sharing = false) {
 
@@ -162,4 +162,4 @@ class IConfig {
 
 	}
 
-}
+}

Zotlabs/Lib/JSalmon.php

@@ -52,7 +52,7 @@ class JSalmon {
 			. base64url_encode($x['alg'],true);
 
 		$key = HTTPSig::get_key(EMPTY_STR,'zot6',base64url_decode($x['sigs']['key_id']));
-		 logger('key: ' . print_r($key,true));
+		logger('key: ' . print_r($key,true), LOGGER_DATA);
 		if($key['portable_id'] && $key['public_key']) {
 			if(Crypto::verify($signed_data,base64url_decode($x['sigs']['value']),$key['public_key'])) {
 				logger('verified');

Zotlabs/Lib/LDSignatures.php

@@ -75,22 +75,23 @@ class LDSignatures {
 	}
 
 	static function hash($obj) {
-
-		return hash('sha256',self::normalise($obj));
+		return hash('sha256', self::normalise($obj));
 	}
 
 	static function normalise($data) {
+		$ret = '';
+
 		if(is_string($data)) {
 			$data = json_decode($data);
 		}
 
 		if(! is_object($data))
-			return '';
+			return $ret;
 
 		jsonld_set_document_loader('jsonld_document_loader');
 
 		try {
-			$d = jsonld_normalize($data,[ 'algorithm' => 'URDNA2015', 'format' => 'application/nquads' ]);
+			$ret = jsonld_normalize($data,[ 'algorithm' => 'URDNA2015', 'format' => 'application/nquads' ]);
 		}
 		catch (\Exception $e) {
 			// Don't log the exception - this can exhaust memory
@@ -98,7 +99,7 @@ class LDSignatures {
 			logger('normalise error: ' . print_r($data,true));
 		}
 
-		return $d;
+		return $ret;
 	}
 
 	static function salmon_sign($data,$channel) {

Zotlabs/Lib/Libsync.php

@@ -135,14 +135,13 @@ class Libsync {
 				$info['collection_members'] = $r;
 		}
 
-		$interval = ((get_config('system', 'delivery_interval') !== false)
-			? intval(get_config('system', 'delivery_interval')) : 2);
+		$interval = get_config('queueworker', 'queue_interval', 500000);
 
 		logger('Packet: ' . print_r($info, true), LOGGER_DATA, LOG_DEBUG);
 
 		$total = count($synchubs);
 		foreach ($synchubs as $hub) {
-			$hash = random_string();
+			$hash = new_uuid();
 			$n    = Libzot::build_packet($channel, 'sync', $env_recips, json_encode($info), 'hz', $hub['hubloc_sitekey'], $hub['site_crypto']);
 			Queue::insert([
 				'hash'       => $hash,
@@ -155,19 +154,26 @@ class Libsync {
 			]);
 
 
+			/*
 			$x = q("select count(outq_hash) as total from outq where outq_delivered = 0");
+
 			if (intval($x[0]['total']) > intval(get_config('system', 'force_queue_threshold', 3000))) {
 				logger('immediate delivery deferred.', LOGGER_DEBUG, LOG_INFO);
 				Queue::update($hash);
 				continue;
 			}
-
+			*/
 
 			Master::Summon(['Deliver', $hash]);
-			$total = $total - 1;
 
-			if ($interval && $total)
-				@time_sleep_until(microtime(true) + (float)$interval);
+			/*
+			$total = $total - 1;
+			*/
+
+			if ($interval) {
+				usleep($interval);
+			}
+
 		}
 	}
 
@@ -192,7 +198,10 @@ class Libsync {
 				dbesc($sender)
 			);
 
-			$DR = new DReport(z_root(), $sender, $d, 'sync');
+			$mid = 'sync';
+
+
+			$DR = new DReport(z_root(), $sender, $d, $mid);
 
 			if (!$r) {
 				$DR->update('recipient not found');
@@ -202,6 +211,7 @@ class Libsync {
 
 			$channel = $r[0];
 
+
 			$DR->set_name($channel['channel_name'] . ' <' . channel_reddress($channel) . '>');
 
 			$max_friends = service_class_fetch($channel['channel_id'], 'total_channels');
@@ -293,8 +303,10 @@ class Libsync {
 			if (array_key_exists('event_item', $arr) && $arr['event_item'])
 				sync_items($channel, $arr['event_item'], ((array_key_exists('relocate', $arr)) ? $arr['relocate'] : null));
 
-			if (array_key_exists('item', $arr) && $arr['item'])
+			if (array_key_exists('item', $arr) && $arr['item']) {
 				sync_items($channel, $arr['item'], ((array_key_exists('relocate', $arr)) ? $arr['relocate'] : null));
+				$mid = $arr['item'][0]['message_id'] . '#sync';
+			}
 
 			// deprecated, maintaining for a few months for upward compatibility
 			// this should sync webpages, but the logic is a bit subtle
@@ -316,7 +328,7 @@ class Libsync {
 				$remote_channel               = $arr['channel'];
 				$remote_channel['channel_id'] = $channel['channel_id'];
 
-				if (array_key_exists('channel_pageflags', $arr['channel']) && intval($arr['channel']['channel_pageflags'])) {
+				if (array_key_exists('channel_pageflags', $arr['channel'])) {
 
 					// Several pageflags are site-specific and cannot be sync'd.
 					// Only allow those bits which are shareable from the remote and then
@@ -740,13 +752,12 @@ class Libsync {
 			 */
 			call_hooks('process_channel_sync_delivery', $addon);
 
-			$DR = new DReport(z_root(), $d, $d, 'sync', 'channel sync delivered');
+			$DR = new DReport(z_root(), $d, $d, $mid, 'channel sync processed');
 
 			$DR->set_name($channel['channel_name'] . ' <' . channel_reddress($channel) . '>');
 
 			$result[] = $DR->get();
 		}
-
 		return $result;
 	}
 
@@ -755,13 +766,19 @@ class Libsync {
 	 *
 	 * @param array $sender
 	 * @param array $arr
-	 * @param boolean $absolute (optional) default false
 	 * @return array
 	 */
 
-	static function sync_locations($sender, $arr, $absolute = false) {
+	static function sync_locations($sender, $arr) {
 
-		$ret = [];
+		$ret = [
+			'change_message' => '',
+			'changed' => false,
+			'message' => ''
+		];
+
+		$what = '';
+		$changed = false;
 
 		// If a sender reports that the channel has been deleted, delete its hubloc
 		if (isset($arr['deleted_locally']) && intval($arr['deleted_locally'])) {
@@ -772,12 +789,9 @@ class Libsync {
 			);
 		}
 
-		if ($arr['locations']) {
+		if (isset($arr['locations']) && $arr['locations']) {
 
-			if ($absolute)
-				Libzot::check_location_move($sender['hash'], $arr['locations']);
-
-			$xisting = q("select * from hubloc where hubloc_hash = '%s'",
+			$xisting = q("select * from hubloc where hubloc_hash = '%s' and hubloc_deleted = 0",
 				dbesc($sender['hash'])
 			);
 
@@ -849,7 +863,9 @@ class Libsync {
 						);
 					}
 
-					// update connection timestamp if this is the site we're talking to
+					// Update connection timestamp if this is the site we're talking to.
+					// Also mark all entries from the current site with different sitekeys
+					// deleted (the site has been re-installed)
 					// This only happens when called from import_xchan
 
 					$current_site = false;
@@ -863,6 +879,12 @@ class Libsync {
 							intval($r[0]['hubloc_id']),
 							dbesc($t)
 						);
+
+						q("update hubloc set hubloc_error = 1, hubloc_deleted = 1 where hubloc_url = '%s' and hubloc_sitekey != '%s'",
+							dbesc($r[0]['hubloc_url']),
+							dbesc($r[0]['hubloc_sitekey'])
+						);
+
 						$current_site = true;
 					}
 
@@ -912,14 +934,7 @@ class Libsync {
 						$what    .= 'primary_hub ';
 						$changed = true;
 					}
-					elseif ($absolute) {
-						// Absolute sync - make sure the current primary is correctly reflected in the xchan
-						$pr = hubloc_change_primary($r[0]);
-						if ($pr) {
-							$what    .= 'xchan_primary ';
-							$changed = true;
-						}
-					}
+
 					if (intval($r[0]['hubloc_deleted']) && (!intval($location['deleted']))) {
 						q("update hubloc set hubloc_deleted = 0, hubloc_updated = '%s' where hubloc_id_url = '%s'",
 							dbesc(datetime_convert()),
@@ -988,9 +1003,9 @@ class Libsync {
 				}
 			}
 
-			// get rid of any hubs we have for this channel which weren't reported.
+			// get rid of any hublocs we have for this channel which weren't reported.
 
-			if ($absolute && $xisting) {
+			if ($xisting) {
 				foreach ($xisting as $x) {
 					if (!array_key_exists('updated', $x)) {
 						logger('Deleting unreferenced hub location ' . $x['hubloc_addr']);

Zotlabs/Lib/Libzotdir.php

@@ -145,8 +145,8 @@ class Libzotdir {
 		if(! $directory_sort_order)
 			$directory_sort_order = 'date';
 
-		$current_order = (($_REQUEST['order']) ? $_REQUEST['order'] : $directory_sort_order);
-		$suggest = (($_REQUEST['suggest']) ? '&suggest=' . $_REQUEST['suggest'] : '');
+		$current_order = $_REQUEST['order'] ?? $directory_sort_order;
+		$suggest = ((isset($_REQUEST['suggest'])) ? '&suggest=' . $_REQUEST['suggest'] : '');
 
 		$url = 'directory?f=';
 
@@ -172,13 +172,12 @@ class Libzotdir {
 	}
 
 	/**
-	 * @brief Checks the directory mode of this hub.
+	 * @brief fetches updates from known directories
 	 *
 	 * Checks the directory mode of this hub to see if it is some form of directory server. If it is,
 	 * get the directory realm of this hub. Fetch a list of all other directory servers in this realm and request
-	 * a directory sync packet. This will contain both directory updates and new ratings. Store these all in the DB.
-	 * In the case of updates, we will query each of them asynchronously from a poller task. Ratings are stored
-	 * directly if the rater's signature matches.
+	 * a directory sync packet. Store these all in the DB.
+	 * In the case of updates, we will query each of them asynchronously from a poller task.
 	 *
 	 * @param int $dirmode;
 	 */
@@ -233,6 +232,8 @@ class Libzotdir {
 		if (! $r)
 			return;
 
+		$dir_trusted_hosts = array_merge(get_directory_fallback_servers(), get_config('system', 'trusted_directory_servers', []));
+
 		foreach ($r as $rr) {
 			if (! $rr['site_directory'])
 				continue;
@@ -264,31 +265,65 @@ class Libzotdir {
 
 			if (is_array($j['transactions']) && count($j['transactions'])) {
 				foreach ($j['transactions'] as $t) {
-
-					if (empty($t['hash']) || empty($t['transaction_id']) || empty($t['address'])) {
+					if (empty($t['hash']) || empty($t['host']) || empty($t['address'])) {
 						continue;
 					}
 
-					$r = q("select * from updates where ud_guid = '%s' limit 1",
-						dbesc($t['transaction_id'])
+					$r = q("select * from updates where ud_hash = '%s' limit 1",
+						dbesc($t['hash'])
 					);
-					if($r)
-						continue;
 
-					$ud_flags = 0;
-					if (is_array($t['flags']) && in_array('deleted',$t['flags']))
-						$ud_flags |= UPDATE_FLAGS_DELETED;
-					if (is_array($t['flags']) && in_array('forced',$t['flags']))
-						$ud_flags |= UPDATE_FLAGS_FORCED;
+					if ($r) {
+						$update = 0;
 
-					$z = q("insert into updates ( ud_hash, ud_guid, ud_date, ud_flags, ud_addr )
-						values ( '%s', '%s', '%s', %d, '%s' ) ",
-						dbesc($t['hash']),
-						dbesc($t['transaction_id']),
-						dbesc($t['timestamp']),
-						intval($ud_flags),
-						dbesc($t['address'])
-					);
+						// no need to look at updates that originated from our own site
+						if ($t['host'] === z_root()) {
+							continue;
+						}
+
+						// there is more recent xchan information
+						if ($r[0]['ud_date'] <= $t['timestamp']) {
+							$update = 1;
+						}
+
+						// the host is trusted and flags have changed - update flags immediately
+						if (in_array($t['host'], $dir_trusted_hosts) &&
+							$rr['site_url'] === $t['host'] &&
+							intval($r[0]['ud_flags']) !== intval($t['flags'])) {
+
+							q("UPDATE updates SET ud_update = %d, ud_flags = %d WHERE ud_id = %d",
+								intval($update),
+								intval($t['flags']),
+								dbesc($r[0]['ud_id'])
+							);
+
+							q("UPDATE xchan SET xchan_censored = %d WHERE xchan_hash = '%s'",
+								intval($t['flags']),
+								dbesc($r[0]['ud_hash'])
+							);
+
+							continue;
+						}
+
+						if (!$update) {
+							continue;
+						}
+
+						q("UPDATE updates SET ud_update = %d WHERE ud_id = %d",
+							intval($update),
+							dbesc($r[0]['ud_id'])
+						);
+					}
+					else {
+						q("insert into updates ( ud_hash, ud_host, ud_date, ud_addr, ud_update, ud_flags )
+							values ( '%s', '%s', '%s', '%s', 1, %d) ",
+							dbesc($t['hash']),
+							dbesc($t['host']),
+							dbesc($t['timestamp']),
+							dbesc($t['address']),
+							dbesc(in_array($t['host'], $dir_trusted_hosts) ? $t['flags'] : 0)
+						);
+					}
 				}
 			}
 		}
@@ -303,8 +338,9 @@ class Libzotdir {
 	 *
 	 * Ignore updating records marked as deleted.
 	 *
-	 * If successful, sets ud_last in the DB to the current datetime for this
+	 * If successful, sets ud_updated in the DB to the current datetime for this
 	 * reddress/webbie.
+	 * Else update ud_last so we can stop trying after 7 days (Daemon/Poller.php)
 	 *
 	 * @param array $ud Entry from update table
 	 */
@@ -313,32 +349,44 @@ class Libzotdir {
 
 		logger('update_directory_entry: ' . print_r($ud,true), LOGGER_DATA);
 
-		if ($ud['ud_addr'] && (! ($ud['ud_flags'] & UPDATE_FLAGS_DELETED))) {
-			$success = false;
-			$zf = [];
+		// TODO: remove this check after all directory servers have version > 8.4
+		// ud_addr will always be the channel url at that time
+		$href = ((strpos($ud['ud_addr'], '://') === false) ? Webfinger::zot_url(punify($ud['ud_addr'])) : punify($ud['ud_addr']));
+		if($href) {
+			$zf = Zotfinger::exec($href);
+			if($zf && array_path_exists('signature/signer',$zf) && $zf['signature']['signer'] === $href && intval($zf['signature']['header_valid'])) {
+				$xc = Libzot::import_xchan($zf['data']);
+
+				// xchan_hash mismatch - this can happen after a site re-install at the same url
+				if ($xc['success'] && $xc['hash'] !== $ud['ud_hash']) {
+					self::delete_by_hash($ud['ud_hash']);
+				}
+
+				// if the channel was deleted - delete the entry in updates
+				if (!empty($zf['data']['deleted_locally'])) {
+					self::delete_by_hash($ud['ud_hash']);
+				}
 
-			$href = Webfinger::zot_url(punify($ud['ud_addr']));
-			if($href) {
-				$zf = Zotfinger::exec($href);
-			}
-			if(array_path_exists('signature/signer',$zf) && $zf['signature']['signer'] === $href && intval($zf['signature']['header_valid'])) {
-				$xc = Libzot::import_xchan($zf['data'], 0, $ud);
 				// This is a workaround for a missing xchan_updated column
 				// TODO: implement xchan_updated in the xchan table and update this column instead
-				if($zf['data']['primary_location']['address'] && $zf['data']['primary_location']['url']) {
+				if(!empty($zf['data']['primary_location']['url'])) {
 					q("UPDATE hubloc SET hubloc_updated = '%s' WHERE hubloc_id_url = '%s' AND hubloc_primary = 1",
 						dbesc(datetime_convert()),
 						dbesc($zf['data']['primary_location']['url'])
 					);
 				}
-			}
-			else {
-				q("update updates set ud_last = '%s' where ud_addr = '%s'",
-					dbesc(datetime_convert()),
-					dbesc($ud['ud_addr'])
-				);
+
+				return true;
 			}
 		}
+
+		q("UPDATE updates SET ud_addr = '%s', ud_last = '%s' WHERE ud_hash = '%s'",
+			dbesc($href ? $href : $ud['ud_addr']),
+			dbesc(datetime_convert()),
+			dbesc($ud['ud_hash'])
+		);
+
+		return false;
 	}
 
 
@@ -353,85 +401,78 @@ class Libzotdir {
 	 */
 
 	static function local_dir_update($uid, $force) {
+		logger('local_dir_update uid: ' . $uid, LOGGER_DEBUG);
 
-
-		logger('local_dir_update: uid: ' . $uid, LOGGER_DEBUG);
-
-		$p = q("select channel.channel_hash, channel_address, channel_timezone, profile.* from profile left join channel on channel_id = uid where uid = %d and is_default = 1",
+		$p = q("select channel.channel_hash, channel_address, channel_timezone, profile.*, xchan.xchan_hidden, xchan.xchan_url from profile left join channel on channel_id = uid left join xchan on channel_hash = xchan_hash where profile.uid = %d and profile.is_default = 1",
 			intval($uid)
 		);
 
-		$profile = array();
-		$profile['encoding'] = 'zot';
-
-		if ($p) {
-			$hash = $p[0]['channel_hash'];
-
-			$profile['description'] = $p[0]['pdesc'];
-			$profile['birthday']    = $p[0]['dob'];
-			if ($age = age($p[0]['dob'],$p[0]['channel_timezone'],''))
-				$profile['age'] = $age;
-
-			$profile['gender']      = $p[0]['gender'];
-			$profile['marital']     = $p[0]['marital'];
-			$profile['sexual']      = $p[0]['sexual'];
-			$profile['locale']      = $p[0]['locality'];
-			$profile['region']      = $p[0]['region'];
-			$profile['postcode']    = $p[0]['postal_code'];
-			$profile['country']     = $p[0]['country_name'];
-			$profile['about']       = $p[0]['about'];
-			$profile['homepage']    = $p[0]['homepage'];
-			$profile['hometown']    = $p[0]['hometown'];
-
-			if ($p[0]['keywords']) {
-				$tags = array();
-				$k = explode(' ', $p[0]['keywords']);
-				if ($k)
-					foreach ($k as $kk)
-						if (trim($kk))
-							$tags[] = trim($kk);
-
-				if ($tags)
-					$profile['keywords'] = $tags;
-			}
-
-			$hidden = (1 - intval($p[0]['publish']));
-
-			logger('hidden: ' . $hidden);
-
-			$r = q("select xchan_hidden from xchan where xchan_hash = '%s'",
-				dbesc($p[0]['channel_hash'])
-			);
-
-			if(intval($r[0]['xchan_hidden']) != $hidden) {
-				$r = q("update xchan set xchan_hidden = %d where xchan_hash = '%s'",
-					intval($hidden),
-					dbesc($hash)
-				);
-			}
-
-			$arr = [ 'channel_id' => $uid, 'hash' => $hash, 'profile' => $profile ];
-			call_hooks('local_dir_update', $arr);
-
-			$address = channel_reddress($p[0]);
-
-			if (perm_is_allowed($uid, '', 'view_profile')) {
-				self::import_directory_profile($hash, $arr['profile'], $address, 0);
-			}
-			else {
-				// they may have made it private
-				q("delete from xprof where xprof_hash = '%s'",
-					dbesc($hash)
-				);
-				q("delete from xtag where xtag_hash = '%s'",
-					dbesc($hash)
-				);
-			}
-
+		if (!$p) {
+			logger('profile not found');
+			return;
 		}
 
-		$ud_hash = random_string() . '@' . \App::get_hostname();
-		self::update_modtime($hash, $ud_hash, channel_reddress($p[0]),(($force) ? UPDATE_FLAGS_FORCED : UPDATE_FLAGS_UPDATED));
+		$profile = [];
+		$profile['encoding'] = 'zot';
+
+		$hash = $p[0]['channel_hash'];
+
+		$profile['description'] = $p[0]['pdesc'];
+		$profile['birthday']    = $p[0]['dob'];
+		if ($age = age($p[0]['dob'],$p[0]['channel_timezone'],''))
+			$profile['age'] = $age;
+
+		$profile['gender']      = $p[0]['gender'];
+		$profile['marital']     = $p[0]['marital'];
+		$profile['sexual']      = $p[0]['sexual'];
+		$profile['locale']      = $p[0]['locality'];
+		$profile['region']      = $p[0]['region'];
+		$profile['postcode']    = $p[0]['postal_code'];
+		$profile['country']     = $p[0]['country_name'];
+		$profile['about']       = $p[0]['about'];
+		$profile['homepage']    = $p[0]['homepage'];
+		$profile['hometown']    = $p[0]['hometown'];
+
+		if ($p[0]['keywords']) {
+			$tags = array();
+			$k = explode(' ', $p[0]['keywords']);
+			if ($k)
+				foreach ($k as $kk)
+					if (trim($kk))
+						$tags[] = trim($kk);
+
+			if ($tags)
+				$profile['keywords'] = $tags;
+		}
+
+		$hidden = (1 - intval($p[0]['publish']));
+
+		logger('hidden: ' . $hidden);
+
+		if(intval($p[0]['xchan_hidden']) !== $hidden) {
+			q("update xchan set xchan_hidden = %d where xchan_hash = '%s'",
+				intval($hidden),
+				dbesc($hash)
+			);
+		}
+
+		$arr = [ 'channel_id' => $uid, 'hash' => $hash, 'profile' => $profile ];
+		call_hooks('local_dir_update', $arr);
+
+		if (perm_is_allowed($uid, '', 'view_profile')) {
+			self::import_directory_profile($hash, $arr['profile']);
+		}
+		else {
+			// they may have made it private
+			q("delete from xprof where xprof_hash = '%s'",
+				dbesc($hash)
+			);
+			q("delete from xtag where xtag_hash = '%s'",
+				dbesc($hash)
+			);
+		}
+
+		self::update($hash, $p[0]['xchan_url']);
 	}
 
 
@@ -441,39 +482,48 @@ class Libzotdir {
 	 *
 	 * @param string $hash
 	 * @param array $profile
-	 * @param string $addr
-	 * @param number $ud_flags (optional) UPDATE_FLAGS_UPDATED
-	 * @param number $suppress_update (optional) default 0
 	 * @return boolean $updated if something changed
 	 */
 
-	static function import_directory_profile($hash, $profile, $addr, $ud_flags = UPDATE_FLAGS_UPDATED, $suppress_update = 0) {
+	static function import_directory_profile($hash, $profile) {
 
 		logger('import_directory_profile', LOGGER_DEBUG);
 		if (! $hash)
 			return false;
 
-		$arr = array();
+		$arr = [];
 
-		$arr['xprof_hash']         = $hash;
-		$arr['xprof_dob']          = (($profile['birthday'] === '0000-00-00') ? $profile['birthday'] : datetime_convert('','',$profile['birthday'],'Y-m-d')); // !!!! check this for 0000 year
-		$arr['xprof_age']          = (($profile['age'])         ? intval($profile['age']) : 0);
-		$arr['xprof_desc']         = (($profile['description']) ? htmlspecialchars($profile['description'], ENT_COMPAT,'UTF-8',false) : '');
-		$arr['xprof_gender']       = (($profile['gender'])      ? htmlspecialchars($profile['gender'],      ENT_COMPAT,'UTF-8',false) : '');
-		$arr['xprof_marital']      = (($profile['marital'])     ? htmlspecialchars($profile['marital'],     ENT_COMPAT,'UTF-8',false) : '');
-		$arr['xprof_sexual']       = (($profile['sexual'])      ? htmlspecialchars($profile['sexual'],      ENT_COMPAT,'UTF-8',false) : '');
-		$arr['xprof_locale']       = (($profile['locale'])      ? htmlspecialchars($profile['locale'],      ENT_COMPAT,'UTF-8',false) : '');
-		$arr['xprof_region']       = (($profile['region'])      ? htmlspecialchars($profile['region'],      ENT_COMPAT,'UTF-8',false) : '');
-		$arr['xprof_postcode']     = (($profile['postcode'])    ? htmlspecialchars($profile['postcode'],    ENT_COMPAT,'UTF-8',false) : '');
-		$arr['xprof_country']      = (($profile['country'])     ? htmlspecialchars($profile['country'],     ENT_COMPAT,'UTF-8',false) : '');
-		$arr['xprof_about']        = (($profile['about'])       ? htmlspecialchars($profile['about'],       ENT_COMPAT,'UTF-8',false) : '');
-		$arr['xprof_homepage']     = (($profile['homepage'])    ? htmlspecialchars($profile['homepage'],    ENT_COMPAT,'UTF-8',false) : '');
-		$arr['xprof_hometown']     = (($profile['hometown'])    ? htmlspecialchars($profile['hometown'],    ENT_COMPAT,'UTF-8',false) : '');
+		$arr['xprof_hash'] = $hash;
+		$arr['xprof_dob'] = '0000-00-00';
+
+		if (isset($profile['birthday'])) {
+			$arr['xprof_dob'] = (($profile['birthday'] === '0000-00-00')
+			? $profile['birthday']
+			: datetime_convert('', '', $profile['birthday'], 'Y-m-d')); // !!!! check this for 0000 year
+		}
+
+		$arr['xprof_age']          = ((isset($profile['age']) && $profile['age']) ? intval($profile['age']) : 0);
+		$arr['xprof_desc']         = ((isset($profile['description']) && $profile['description']) ? htmlspecialchars($profile['description'], ENT_COMPAT,'UTF-8',false) : '');
+		$arr['xprof_gender']       = ((isset($profile['gender']) && $profile['gender'])     ? htmlspecialchars($profile['gender'],      ENT_COMPAT,'UTF-8',false) : '');
+		$arr['xprof_marital']      = ((isset($profile['marital']) && $profile['marital'])   ? htmlspecialchars($profile['marital'],     ENT_COMPAT,'UTF-8',false) : '');
+		$arr['xprof_sexual']       = ((isset($profile['sexual']) && $profile['sexual'])     ? htmlspecialchars($profile['sexual'],      ENT_COMPAT,'UTF-8',false) : '');
+		$arr['xprof_locale']       = ((isset($profile['locale']) && $profile['locale'])     ? htmlspecialchars($profile['locale'],      ENT_COMPAT,'UTF-8',false) : '');
+		$arr['xprof_region']       = ((isset($profile['region']) && $profile['region'])     ? htmlspecialchars($profile['region'],      ENT_COMPAT,'UTF-8',false) : '');
+		$arr['xprof_postcode']     = ((isset($profile['postcode']) && $profile['postcode']) ? htmlspecialchars($profile['postcode'],    ENT_COMPAT,'UTF-8',false) : '');
+		$arr['xprof_country']      = ((isset($profile['country']) && $profile['country'])   ? htmlspecialchars($profile['country'],     ENT_COMPAT,'UTF-8',false) : '');
+		$arr['xprof_about']        = ((isset($profile['about']) && $profile['about'])       ? htmlspecialchars($profile['about'],       ENT_COMPAT,'UTF-8',false) : '');
+		$arr['xprof_homepage']     = ((isset($profile['homepage']) && $profile['homepage']) ? htmlspecialchars($profile['homepage'],    ENT_COMPAT,'UTF-8',false) : '');
+		$arr['xprof_hometown']     = ((isset($profile['hometown']) && $profile['hometown']) ? htmlspecialchars($profile['hometown'],    ENT_COMPAT,'UTF-8',false) : '');
 
 		$clean = array();
 		if (array_key_exists('keywords', $profile) and is_array($profile['keywords'])) {
 			self::import_directory_keywords($hash,$profile['keywords']);
+
 			foreach ($profile['keywords'] as $kw) {
+				if (in_array($kw, $clean)) {
+					continue;
+				}
+
 				$kw = trim(htmlspecialchars($kw,ENT_COMPAT, 'UTF-8', false));
 				$kw = trim($kw, ',');
 				$clean[] = $kw;
@@ -580,9 +630,6 @@ class Libzotdir {
 		 */
 		call_hooks('import_directory_profile', $d);
 
-		if (($d['update']) && (! $suppress_update))
-			self::update_modtime($arr['xprof_hash'],random_string() . '@' . \App::get_hostname(), $addr, $ud_flags);
-
 		return $d['update'];
 	}
 
@@ -600,6 +647,10 @@ class Libzotdir {
 			dbesc($hash)
 		);
 
+		$xchan = q("select xchan_censored from xchan where xchan_hash = '%s'",
+			dbesc($hash)
+		);
+
 		if($r) {
 			foreach($r as $rr)
 				$existing[] = $rr['xtag_term'];
@@ -607,6 +658,10 @@ class Libzotdir {
 
 		$clean = array();
 		foreach($keywords as $kw) {
+			if (in_array($kw, $clean)) {
+				continue;
+			}
+
 			$kw = trim(htmlspecialchars($kw,ENT_COMPAT, 'UTF-8', false));
 			$kw = trim($kw, ',');
 			$clean[] = $kw;
@@ -621,9 +676,10 @@ class Libzotdir {
 		}
 		foreach($clean as $x) {
 			if(! in_array($x, $existing)) {
-				$r = q("insert into xtag ( xtag_hash, xtag_term, xtag_flags) values ( '%s' ,'%s', 0 )",
+				$r = q("insert into xtag ( xtag_hash, xtag_term, xtag_flags) values ( '%s' ,'%s', %d )",
 					dbesc($hash),
-					dbesc($x)
+					dbesc($x),
+					intval($xchan[0]['xchan_censored'])
 				);
 			}
 		}
@@ -633,13 +689,12 @@ class Libzotdir {
 	/**
 	 * @brief
 	 *
-	 * @param string $hash
-	 * @param string $guid
-	 * @param string $addr
-	 * @param int $flags (optional) default 0
+	 * @param string $hash the channel hash
+	 * @param string $addr the channel url
+	 * @param bool $bump_date (optional) default true
 	 */
 
-	static function update_modtime($hash, $guid, $addr, $flags = 0) {
+	static function update($hash, $addr, $bump_date = true, $flag = null) {
 
 		$dirmode = intval(get_config('system', 'directory_mode'));
 
@@ -647,26 +702,70 @@ class Libzotdir {
 			return;
 		}
 
-		if (empty($hash) || empty($guid) || empty($addr)) {
+		if (empty($hash) || empty($addr)) {
 			return;
 		}
 
-		if($flags) {
-			q("insert into updates (ud_hash, ud_guid, ud_date, ud_flags, ud_addr ) values ( '%s', '%s', '%s', %d, '%s' )",
-				dbesc($hash),
-				dbesc($guid),
-				dbesc(datetime_convert()),
-				intval($flags),
-				dbesc($addr)
-			);
+		$u = q("SELECT * FROM updates WHERE ud_hash = '%s' LIMIT 1",
+			dbesc($hash)
+		);
+
+		$date_sql = '';
+		if ($bump_date) {
+			$date_sql = "ud_date = '" . dbesc(datetime_convert()) . "',";
 		}
-		else {
-			q("update updates set ud_flags = ( ud_flags | %d ) where ud_addr = '%s' and not (ud_flags & %d)>0 ",
-				intval(UPDATE_FLAGS_UPDATED),
+
+		$flag_sql = '';
+		if ($flag !== null) {
+			$flag_sql = "ud_flags = '" . intval($flag) . "',";
+		}
+
+
+		if ($u) {
+			$x = q("UPDATE updates SET $date_sql $flag_sql ud_last = '%s', ud_host = '%s', ud_addr = '%s', ud_update = 0 WHERE ud_id = %d",
+				dbesc(NULL_DATE),
+				dbesc(z_root()),
 				dbesc($addr),
-				intval(UPDATE_FLAGS_UPDATED)
+				intval($u[0]['ud_id'])
 			);
+
+			return;
 		}
+
+		q("INSERT INTO updates (ud_hash, ud_host, ud_date, ud_addr, ud_flags) VALUES ( '%s', '%s', '%s', '%s', %d )",
+			dbesc($hash),
+			dbesc(z_root()),
+			dbesc(datetime_convert()),
+			dbesc($addr),
+			intval($flag)
+		);
+
+		return;
+
+	}
+
+
+	/**
+	 * @brief deletes a entry in updates by hash
+	 *
+	 * @param string $hash the channel hash
+	 * @return boolean
+	 */
+
+	static function delete_by_hash($hash) {
+		if (!$hash) {
+			return false;
+		}
+
+		$x = q("DELETE FROM updates WHERE ud_hash = '%s'",
+			dbesc($hash)
+		);
+
+		if ($x) {
+			return true;
+		}
+
+		return false;
 	}
 
 }

Zotlabs/Lib/MessageFilter.php

@@ -2,85 +2,120 @@
 
 namespace Zotlabs\Lib;
 
-
+require_once('include/html2plain.php');
 
 class MessageFilter {
 
+	public static function evaluate($item, $incl, $excl) {
 
-	static public function evaluate($item,$incl,$excl) {
-
-		require_once('include/html2plain.php');
-
-		$text = prepare_text($item['body'],$item['mimetype']);
+		$text = prepare_text($item['body'],((isset($item['mimetype'])) ? $item['mimetype'] : 'text/bbcode'));
 		$text = html2plain(($item['title']) ? $item['title'] . ' ' . $text : $text);
 
-
 		$lang = null;
 
-		if((strpos($incl,'lang=') !== false) || (strpos($excl,'lang=') !== false) || (strpos($incl,'lang!=') !== false) || (strpos($excl,'lang!=') !== false)) {
+		if ((strpos($incl, 'lang=') !== false) || (strpos($excl, 'lang=') !== false) || (strpos($incl, 'lang!=') !== false) || (strpos($excl, 'lang!=') !== false)) {
 			$lang = detect_language($text);
 		}
 
-		$tags = ((is_array($item['term']) && count($item['term'])) ? $item['term'] : false);
+		$tags = ((isset($item['term']) && is_array($item['term']) && count($item['term'])) ? $item['term'] : false);
 
 		// exclude always has priority
 
-		$exclude = (($excl) ? explode("\n",$excl) : null);
+		$exclude = (($excl) ? explode("\n", $excl) : null);
 
-		if($exclude) {
-			foreach($exclude as $word) {
-				$word = trim($word);
-				if(! $word)
+		if ($exclude) {
+			foreach ($exclude as $word) {
+				$word = html_entity_decode(trim($word));
+				if (! $word) {
 					continue;
-				if(substr($word,0,1) === '#' && $tags) {
-					foreach($tags as $t)
-						if((($t['ttype'] == TERM_HASHTAG) || ($t['ttype'] == TERM_COMMUNITYTAG)) && (($t['term'] === substr($word,1)) || (substr($word,1) === '*')))
-							return false;
 				}
-				elseif(substr($word,0,1) === '$' && $tags) {
-					foreach($tags as $t)
-						if(($t['ttype'] == TERM_CATEGORY) && (($t['term'] === substr($word,1)) || (substr($word,1) === '*')))
-							return false;
+				if (isset($lang) && ((strpos($word, 'lang=') === 0) || (strpos($word, 'lang!=') === 0))) {
+					if (!strlen($lang)) {
+						// Result is ambiguous. As we are matching deny rules only at this time, continue tests.
+						// Any matching deny rule concludes testing.
+						continue;
+					}
+					if (strpos($word, 'lang=') === 0 && strcasecmp($lang, trim(substr($word, 5))) == 0) {
+						return false;
+					} elseif (strpos($word, 'lang!=') === 0 && strcasecmp($lang, trim(substr($word, 6))) != 0) {
+						return false;
+					}
 				}
-				elseif((strpos($word,'/') === 0) && preg_match($word,$text))
+                elseif (substr($word, 0, 1) === '#' && $tags) {
+					foreach ($tags as $t) {
+						if ((($t['ttype'] == TERM_HASHTAG) || ($t['ttype'] == TERM_COMMUNITYTAG)) && (($t['term'] === substr($word, 1)) || (substr($word, 1) === '*'))) {
+							return false;
+						}
+					}
+				} elseif (substr($word, 0, 1) === '$' && $tags) {
+					foreach ($tags as $t) {
+						if (($t['ttype'] == TERM_CATEGORY) && (($t['term'] === substr($word, 1)) || (substr($word, 1) === '*'))) {
+							return false;
+						}
+					}
+				} elseif (substr($word, 0, 2) === '?+') {
+					if (self::test_condition(substr($word, 2), $item['obj'])) {
+						return false;
+					}
+				} elseif (substr($word, 0, 1) === '?') {
+					if (self::test_condition(substr($word, 1), $item)) {
+						return false;
+					}
+				} elseif ((strpos($word, '/') === 0) && preg_match($word, $text)) {
 					return false;
-				elseif((strpos($word,'lang=') === 0) && ($lang) && (strcasecmp($lang,trim(substr($word,5))) == 0))
-					return false;
-				elseif((strpos($word,'lang!=') === 0) && ($lang) && (strcasecmp($lang,trim(substr($word,6))) != 0))
-					return false;
-				elseif(stristr($text,$word) !== false)
+				} elseif (stristr($text, $word) !== false) {
 					return false;
+				}
 			}
 		}
 
-		$include = (($incl) ? explode("\n",$incl) : null);
+		$include = (($incl) ? explode("\n", $incl) : null);
 
-		if($include) {
-			foreach($include as $word) {
-				$word = trim($word);
-				if(! $word)
+		if ($include) {
+			foreach ($include as $word) {
+				$word = html_entity_decode(trim($word));
+				if (! $word) {
 					continue;
-				if(substr($word,0,1) === '#' && $tags) {
-					foreach($tags as $t)
-						if((($t['ttype'] == TERM_HASHTAG) || ($t['ttype'] == TERM_COMMUNITYTAG)) && (($t['term'] === substr($word,1)) || (substr($word,1) === '*')))
-							return true;
 				}
-				elseif(substr($word,0,1) === '$' && $tags) {
-					foreach($tags as $t)
-						if(($t['ttype'] == TERM_CATEGORY) && (($t['term'] === substr($word,1)) || (substr($word,1) === '*')))
-							return true;
+				if (isset($lang) && ((strpos($word, 'lang=') === 0) || (strpos($word, 'lang!=') === 0))) {
+					if (!strlen($lang))  {
+						// Result is ambiguous. However we are checking allow rules
+						// and an ambiguous language is always permitted.
+						return true;
+					}
+					if (strpos($word, 'lang=') === 0 && strcasecmp($lang, trim(substr($word, 5))) == 0) {
+						return true;
+					} elseif (strpos($word, 'lang!=') === 0 && strcasecmp($lang, trim(substr($word, 6))) != 0) {
+						return true;
+					}
 				}
-				elseif((strpos($word,'/') === 0) && preg_match($word,$text))
+                elseif (substr($word, 0, 1) === '#' && $tags) {
+					foreach ($tags as $t) {
+						if ((($t['ttype'] == TERM_HASHTAG) || ($t['ttype'] == TERM_COMMUNITYTAG)) && (($t['term'] === substr($word, 1)) || (substr($word, 1) === '*'))) {
+							return true;
+						}
+					}
+				} elseif (substr($word, 0, 1) === '$' && $tags) {
+					foreach ($tags as $t) {
+						if (($t['ttype'] == TERM_CATEGORY) && (($t['term'] === substr($word, 1)) || (substr($word, 1) === '*'))) {
+							return true;
+						}
+					}
+				} elseif (substr($word, 0, 2) === '?+') {
+					if (self::test_condition(substr($word, 2), $item['obj'])) {
+						return true;
+					}
+				} elseif (substr($word, 0, 1) === '?') {
+					if (self::test_condition(substr($word, 1), $item)) {
+						return true;
+					}
+				} elseif ((strpos($word, '/') === 0) && preg_match($word, $text)) {
 					return true;
-				elseif((strpos($word,'lang=') === 0) && ($lang) && (strcasecmp($lang,trim(substr($word,5))) == 0))
-					return true;
-				elseif((strpos($word,'lang!=') === 0) && ($lang) && (strcasecmp($lang,trim(substr($word,6))) != 0))
-					return true;
-				elseif(stristr($text,$word) !== false)
+				} elseif (stristr($text, $word) !== false) {
 					return true;
+				}
 			}
-		}
-		else {
+		} else {
 			return true;
 		}
 
@@ -88,4 +123,123 @@ class MessageFilter {
 	}
 
 
+	/**
+	 * @brief Test for Conditional Execution conditions. Shamelessly ripped off from Code/Render/Comanche
+	 *
+	 * This is extensible. The first version of variable testing supports tests of the forms:
+	 *
+	 * - ?foo ~= baz which will check if item.foo contains the string 'baz';
+	 * - ?foo == baz which will check if item.foo is the string 'baz';
+	 * - ?foo != baz which will check if item.foo is not the string 'baz';
+	 * - ?foo >= 3 which will check if item.foo is greater than or equal to 3;
+	 * - ?foo > 3 which will check if item.foo is greater than 3;
+	 * - ?foo <= 3 which will check if item.foo is less than or equal to 3;
+	 * - ?foo < 3 which will check if item.foo is less than 3;
+	 *
+	 * - ?foo {} baz which will check if 'baz' is an array element in item.foo
+	 * - ?foo {*} baz which will check if 'baz' is an array key in item.foo
+	 * - ?foo which will check for a return of a true condition for item.foo;
+     * - ?!foo which will check for a return of a false condition for item.foo;
+     *
+	 * The values 0, '', an empty array, and an unset value will all evaluate to false.
+	 *
+	 * @param string $s
+	 * @param array $item
+	 * @return bool
+	 */
+
+	public static function test_condition($s,$item) {
+
+		if (preg_match('/(.*?)\s\~\=\s(.*?)$/', $s, $matches)) {
+			$x = ((array_key_exists(trim($matches[1]),$item)) ? $item[trim($matches[1])] : EMPTY_STR);
+			if (stripos($x, trim($matches[2])) !== false) {
+				return true;
+			}
+			return false;
+		}
+
+		if (preg_match('/(.*?)\s\=\=\s(.*?)$/', $s, $matches)) {
+			$x = ((array_key_exists(trim($matches[1]),$item)) ? $item[trim($matches[1])] : EMPTY_STR);
+			if ($x == trim($matches[2])) {
+				return true;
+			}
+			return false;
+		}
+
+		if (preg_match('/(.*?)\s\!\=\s(.*?)$/', $s, $matches)) {
+			$x = ((array_key_exists(trim($matches[1]),$item)) ? $item[trim($matches[1])] : EMPTY_STR);
+			if ($x != trim($matches[2])) {
+				return true;
+			}
+			return false;
+		}
+
+		if (preg_match('/(.*?)\s\>\=\s(.*?)$/', $s, $matches)) {
+			$x = ((array_key_exists(trim($matches[1]),$item)) ? $item[trim($matches[1])] : EMPTY_STR);
+			if ($x >= trim($matches[2])) {
+				return true;
+			}
+			return false;
+		}
+
+		if (preg_match('/(.*?)\s\<\=\s(.*?)$/', $s, $matches)) {
+			$x = ((array_key_exists(trim($matches[1]),$item)) ? $item[trim($matches[1])] : EMPTY_STR);
+			if ($x <= trim($matches[2])) {
+				return true;
+			}
+			return false;
+		}
+
+		if (preg_match('/(.*?)\s\>\s(.*?)$/', $s, $matches)) {
+			$x = ((array_key_exists(trim($matches[1]),$item)) ? $item[trim($matches[1])] : EMPTY_STR);
+			if ($x > trim($matches[2])) {
+				return true;
+			}
+			return false;
+		}
+
+		if (preg_match('/(.*?)\s\>\s(.*?)$/', $s, $matches)) {
+			$x = ((array_key_exists(trim($matches[1]),$item)) ? $item[trim($matches[1])] : EMPTY_STR);
+			if ($x < trim($matches[2])) {
+				return true;
+			}
+			return false;
+		}
+
+		if (preg_match('/[\$](.*?)\s\{\}\s(.*?)$/', $s, $matches)) {
+			$x = ((array_key_exists(trim($matches[1]),$item)) ? $item[trim($matches[1])] : EMPTY_STR);
+			if (is_array($x) && in_array(trim($matches[2]), $x)) {
+				return true;
+			}
+			return false;
+		}
+
+		if (preg_match('/(.*?)\s\{\*\}\s(.*?)$/', $s, $matches)) {
+			$x = ((array_key_exists(trim($matches[1]),$item)) ? $item[trim($matches[1])] : EMPTY_STR);
+			if (is_array($x) && array_key_exists(trim($matches[2]), $x)) {
+				return true;
+			}
+			return false;
+		}
+
+		// Ordering of this check (for falsiness) with relation to the following one (check for truthiness) is important.
+		if (preg_match('/\!(.*?)$/', $s, $matches)) {
+			$x = ((array_key_exists(trim($matches[1]),$item)) ? $item[trim($matches[1])] : EMPTY_STR);
+			if (!$x) {
+				return true;
+			}
+			return false;
+		}
+
+		if (preg_match('/(.*?)$/', $s, $matches)) {
+			$x = ((array_key_exists(trim($matches[1]),$item)) ? $item[trim($matches[1])] : EMPTY_STR);
+			if ($x) {
+				return true;
+			}
+			return false;
+		}
+
+		return false;
+	}
+
 }

Zotlabs/Lib/NativeWiki.php

@@ -1,322 +0,0 @@
-<?php
-
-namespace Zotlabs\Lib;
-
-use Zotlabs\Lib\Libsync;
-
-define ( 'NWIKI_ITEM_RESOURCE_TYPE', 'nwiki' );
-
-class NativeWiki {
-
-
-	public static function listwikis($channel, $observer_hash) {
-
-		$sql_extra = item_permissions_sql($channel['channel_id'], $observer_hash);
-		$wikis = q("SELECT * FROM item
-			WHERE resource_type = '%s' AND mid = parent_mid AND uid = %d AND item_deleted = 0 $sql_extra",
-			dbesc(NWIKI_ITEM_RESOURCE_TYPE),
-			intval($channel['channel_id'])
-		);
-
-		if($wikis) {
-			foreach($wikis as &$w) {
-
-				$w['json_allow_cid']  = acl2json($w['allow_cid']);
-				$w['json_allow_gid']  = acl2json($w['allow_gid']);
-				$w['json_deny_cid']   = acl2json($w['deny_cid']);
-				$w['json_deny_gid']   = acl2json($w['deny_gid']);
-
-				$w['rawName']  = get_iconfig($w, 'wiki', 'rawName');
-				$w['htmlName'] = escape_tags($w['rawName']);
-				//$w['urlName']  = urlencode(urlencode($w['rawName']));
-				$w['urlName']  = self::name_encode($w['rawName']);
-				$w['mimeType'] = get_iconfig($w, 'wiki', 'mimeType');
-				$w['typelock'] = get_iconfig($w, 'wiki', 'typelock');
-				$w['lockstate']     = (($w['allow_cid'] || $w['allow_gid'] || $w['deny_cid'] || $w['deny_gid']) ? 'lock' : 'unlock');
-			}
-		}
-		// TODO: query db for wikis the observer can access. Return with two lists, for read and write access
-		return array('wikis' => $wikis);
-	}
-
-
-	public static function create_wiki($channel, $observer_hash, $wiki, $acl) {
-
-		$resource_id = new_uuid();
-		$uuid = new_uuid();
-
-		$ac = $acl->get();
-		$mid = z_root() . '/item/' . $uuid;
-
-		$arr = array();	// Initialize the array of parameters for the post
-		$item_hidden = ((intval($wiki['postVisible']) === 0) ? 1 : 0);
-		$wiki_url = z_root() . '/wiki/' . $channel['channel_address'] . '/' . $wiki['urlName'];
-		$arr['aid'] = $channel['channel_account_id'];
-		$arr['uuid'] = $uuid;
-		$arr['uid'] = $channel['channel_id'];
-		$arr['mid'] = $mid;
-		$arr['parent_mid'] = $mid;
-		$arr['item_hidden'] = $item_hidden;
-		$arr['resource_type'] = NWIKI_ITEM_RESOURCE_TYPE;
-		$arr['resource_id'] = $resource_id;
-		$arr['owner_xchan'] = $channel['channel_hash'];
-		$arr['author_xchan'] = $observer_hash;
-		$arr['plink'] = $mid;
-		$arr['llink'] = z_root() . '/display/' . gen_link_id($mid);
-		$arr['title'] = $wiki['htmlName'];  // name of new wiki;
-		$arr['allow_cid'] = $ac['allow_cid'];
-		$arr['allow_gid'] = $ac['allow_gid'];
-		$arr['deny_cid'] = $ac['deny_cid'];
-		$arr['deny_gid'] = $ac['deny_gid'];
-		$arr['item_wall'] = 1;
-		$arr['item_origin'] = 1;
-		$arr['item_thread_top'] = 1;
-		$arr['item_private'] = intval($acl->is_private());
-		$arr['verb'] = ACTIVITY_CREATE;
-		$arr['obj_type'] = 'Document';
-		$arr['body'] = '[table][tr][td][h1]New Wiki[/h1][/td][/tr][tr][td][zrl=' . $wiki_url . ']' . $wiki['htmlName'] . '[/zrl][/td][/tr][/table]';
-
-		$arr['public_policy'] = map_scope(\Zotlabs\Access\PermissionLimits::Get($channel['channel_id'],'view_wiki'),true);
-
-		// Save the wiki name information using iconfig. This is shareable.
-		if(! set_iconfig($arr, 'wiki', 'rawName', $wiki['rawName'], true)) {
-			return array('item' => null, 'success' => false);
-		}
-		if(! set_iconfig($arr, 'wiki', 'mimeType', $wiki['mimeType'], true)) {
-			return array('item' => null, 'success' => false);
-		}
-
-		set_iconfig($arr,'wiki','typelock',$wiki['typelock'],true);
-
-		$post = item_store($arr);
-
-		$item_id = $post['item_id'];
-
-		if($item_id) {
-			\Zotlabs\Daemon\Master::Summon(array('Notifier', 'activity', $item_id));
-			return array('item' => $post['item'], 'item_id' => $item_id, 'success' => true);
-		}
-		else {
-			return array('item' => null, 'success' => false);
-		}
-	}
-
-
-	public static function update_wiki($channel_id, $observer_hash, $arr, $acl) {
-
-		$w = self::get_wiki($channel_id, $observer_hash, $arr['resource_id']);
-		$item = $w['wiki'];
-
-		if(! $item) {
-			return array('item' => null, 'success' => false);
-		}
-
-		$x = $acl->get();
-
-		$item['allow_cid']    = $x['allow_cid'];
-		$item['allow_gid']    = $x['allow_gid'];
-		$item['deny_cid']     = $x['deny_cid'];
-		$item['deny_gid']     = $x['deny_gid'];
-		$item['item_private'] = intval($acl->is_private());
-
-		$update_title = false;
-
-		if($item['title'] !== $arr['updateRawName']) {
-			$update_title = true;
-			$item['title'] = $arr['updateRawName'];
-		}
-
-		$update = item_store_update($item);
-
-		$item_id = $update['item_id'];
-
-		// update acl for any existing wiki pages
-
-		q("update item set allow_cid = '%s', allow_gid = '%s', deny_cid = '%s', deny_gid = '%s', item_private = %d where resource_type = 'nwikipage' and resource_id = '%s'",
-			dbesc($item['allow_cid']),
-			dbesc($item['allow_gid']),
-			dbesc($item['deny_cid']),
-			dbesc($item['deny_gid']),
-			dbesc($item['item_private']),
-			dbesc($arr['resource_id'])
-		);
-
-
-		if($update['item_id']) {
-			info( t('Wiki updated successfully'));
-			if($update_title) {
-				// Update the wiki name information using iconfig.
-				if(! set_iconfig($update['item_id'], 'wiki', 'rawName', $arr['updateRawName'], true)) {
-					return array('item' => null, 'success' => false);
-				}
-			}
-			return array('item' => $update['item'], 'item_id' => $update['item_id'], 'success' => $update['success']);
-		}
-		else {
-			return array('item' => null, 'success' => false);
-		}
-	}
-
-
-	public static function sync_a_wiki_item($uid,$id,$resource_id) {
-
-		$r = q("SELECT * from item WHERE uid = %d AND ( id = %d OR ( resource_type = '%s' and resource_id = '%s' )) ",
-			intval($uid),
-			intval($id),
-			dbesc(NWIKI_ITEM_RESOURCE_TYPE),
-			dbesc($resource_id)
-		);
-		if($r) {
-
-			$q = q("select * from item where resource_type = 'nwikipage' and resource_id = '%s'",
-				dbesc($r[0]['resource_id'])
-			);
-			if($q) {
-				$r = array_merge($r,$q);
-			}
-			xchan_query($r);
-			$sync_item = fetch_post_tags($r);
-			if($sync_item) {
-				$pkt = [];
-				foreach($sync_item as $w) {
-					$pkt[] = encode_item($w,true);
-				}
-				Libsync::build_sync_packet($uid,array('wiki' => $pkt));
-			}
-		}
-	}
-
-
-	public static function delete_wiki($channel_id,$observer_hash,$resource_id) {
-
-		$w = self::get_wiki($channel_id,$observer_hash,$resource_id);
-		if(! $w['wiki']) {
-			return [ 'success' => false ];
-		}
-		else {
-
-			$r = q("SELECT id FROM item WHERE uid = %s AND resource_id = '%s'",
-				intval($channel_id),
-				dbesc($resource_id)
-			);
-
-			$ids = array_column($r, 'id');
-			drop_items($ids, true, DROPITEM_PHASE1);
-
-			info(t('Wiki files deleted successfully'));
-
-			return [ 'success' => true ];
-		}
-	}
-
-
-	public static function get_wiki($channel_id, $observer_hash, $resource_id) {
-
-		$sql_extra = item_permissions_sql($channel_id,$observer_hash);
-
-		$item = q("SELECT * FROM item WHERE uid = %d AND resource_type = '%s' AND resource_id = '%s' AND item_deleted = 0
-			$sql_extra ORDER BY id LIMIT 1",
-			intval($channel_id),
-			dbesc(NWIKI_ITEM_RESOURCE_TYPE),
-			dbesc($resource_id)
-		);
-		if(! $item) {
-			return [ 'wiki' => null ];
-		}
-		else {
-
-			$w = $item[0];	// wiki item table record
-			// Get wiki metadata
-			$rawName  = get_iconfig($w, 'wiki', 'rawName');
-			$mimeType = get_iconfig($w, 'wiki', 'mimeType');
-			$typelock = get_iconfig($w, 'wiki', 'typelock');
-
-			return array(
-				'wiki'     => $w,
-				'rawName'  => $rawName,
-				'htmlName' => escape_tags($rawName),
-				//'urlName'  => urlencode(urlencode($rawName)),
-				'urlName'  => self::name_encode($rawName),
-				'mimeType' => $mimeType,
-				'typelock' => $typelock
-			);
-		}
-	}
-
-
-	public static function exists_by_name($uid, $urlName) {
-
-		$sql_extra = item_permissions_sql($uid);
-
-		$item = q("SELECT item.id, resource_id FROM item left join iconfig on iconfig.iid = item.id
-			WHERE resource_type = '%s' AND iconfig.v = '%s' AND uid = %d
-			AND item_deleted = 0 $sql_extra limit 1",
-			dbesc(NWIKI_ITEM_RESOURCE_TYPE),
-			//dbesc(urldecode($urlName)),
-			dbesc(self::name_decode($urlName)),
-			intval($uid)
-		);
-
-		if($item) {
-			return array('id' => $item[0]['id'], 'resource_id' => $item[0]['resource_id']);
-		}
-		else {
-			return array('id' => null, 'resource_id' => null);
-		}
-	}
-
-
-	public static function get_permissions($resource_id, $owner_id, $observer_hash) {
-
-		// TODO: For now, only the owner can edit
-		$sql_extra = item_permissions_sql($owner_id, $observer_hash);
-
-		if(local_channel() && local_channel() == $owner_id) {
-			return [ 'read' => true, 'write' => true, 'success' => true ];
-		}
-
-		$r = q("SELECT * FROM item WHERE uid = %d and resource_type = '%s' AND resource_id = '%s' $sql_extra LIMIT 1",
-			intval($owner_id),
-			dbesc(NWIKI_ITEM_RESOURCE_TYPE),
-			dbesc($resource_id)
-		);
-
-		if(! $r) {
-			return array('read' => false, 'write' => false, 'success' => true);
-		}
-		else {
-			$write = perm_is_allowed($owner_id, $observer_hash,'write_wiki');
-			return array('read' => true, 'write' => $write, 'success' => true);
-		}
-	}
-
-
-	public static function name_encode ($string) {
-
-		$string = html_entity_decode($string);
-		$encoding = mb_internal_encoding();
-		mb_internal_encoding("UTF-8");
-		$ret = mb_ereg_replace_callback ('[^A-Za-z0-9\-\_\.\~]',function ($char) {
-	                $charhex = unpack('H*',$char[0]);
-                	$ret = '('.$charhex[1].')';
-                	return $ret;
- 			}
-			,$string);
-		mb_internal_encoding($encoding);
-		return $ret;
-	}
-
-
-	public static function name_decode ($string) {
-
-		$encoding = mb_internal_encoding();
-		mb_internal_encoding("UTF-8");
-		$ret = mb_ereg_replace_callback ('(\(([0-9a-f]+)\))',function ($chars) {
-			return pack('H*',$chars[2]);
-			}
-			,$string);
-		mb_internal_encoding($encoding);
-		return $ret;
-	}
-
-}

Zotlabs/Lib/NativeWikiPage.php

@@ -1,725 +0,0 @@
-<?php
-
-namespace Zotlabs\Lib;
-
-use App;
-use Zotlabs\Access\PermissionLimits;
-
-class NativeWikiPage {
-
-	static public function page_list($channel_id, $observer_hash, $resource_id) {
-
-		// TODO: Create item table records for pages so that metadata like title can be applied
-		$w = NativeWiki::get_wiki($channel_id, $observer_hash, $resource_id);
-
-		$pages[] = [
-			'resource_id' => '',
-			'title'       => 'Home',
-			'url'         => 'Home',
-			'link_id'     => 'id_wiki_home_0'
-		];
-
-		$sql_extra = item_permissions_sql($channel_id, $observer_hash);
-
-		$r = q("select * from item where resource_type = 'nwikipage' and resource_id = '%s' and uid = %d and item_deleted = 0
-			$sql_extra order by title asc",
-			dbesc($resource_id),
-			intval($channel_id)
-		);
-		if ($r) {
-			$x = [];
-			$y = [];
-
-			foreach ($r as $rv) {
-				if (!in_array($rv['mid'], $x)) {
-					$y[] = $rv;
-					$x[] = $rv['mid'];
-				}
-			}
-
-			$items = fetch_post_tags($y, true);
-
-			foreach ($items as $page_item) {
-				$title = get_iconfig($page_item['id'], 'nwikipage', 'pagetitle', t('(No Title)'));
-				if (urldecode($title) !== 'Home') {
-					$pages[] = [
-						'resource_id' => $resource_id,
-						'title'       => escape_tags($title),
-						//'url'         => str_replace('%2F','/',urlencode(str_replace('%2F','/',urlencode($title)))),
-						'url'         => NativeWiki::name_encode($title),
-						'link_id'     => 'id_' . substr($resource_id, 0, 10) . '_' . $page_item['id']
-					];
-				}
-			}
-		}
-
-		return ['pages' => $pages, 'wiki' => $w];
-	}
-
-
-	static public function create_page($channel, $observer_hash, $name, $resource_id, $mimetype = 'text/bbcode') {
-
-		logger('mimetype: ' . $mimetype);
-
-		if (!in_array($mimetype, ['text/markdown', 'text/bbcode', 'text/plain', 'text/html']))
-			$mimetype = 'text/markdown';
-
-		$w = NativeWiki::get_wiki($channel['channel_id'], $observer_hash, $resource_id);
-
-		if (!$w['wiki']) {
-			return ['content' => null, 'message' => 'Error reading wiki', 'success' => false];
-		}
-
-		// backslashes won't work well in the javascript functions
-		$name = str_replace('\\', '', $name);
-
-		$uuid = new_uuid();
-		$mid  = z_root() . '/item/' . $uuid;
-
-		// create an empty activity
-		$arr                    = [];
-		$arr['aid']             = $channel['channel_account_id'];
-		$arr['uid']             = $channel['channel_id'];
-		$arr['mid']             = $mid;
-		$arr['parent_mid']      = $w['wiki']['mid'];
-		$arr['parent']          = $w['wiki']['parent'];
-		$arr['uuid']            = $uuid;
-		$arr['item_hidden']     = $w['wiki']['item_hidden'];
-		$arr['plink']           = $mid;
-		$arr['llink']           = z_root() . '/display/' . gen_link_id($mid);
-		$arr['author_xchan']    = $observer_hash;
-		$arr['mimetype']        = $mimetype;
-		$arr['title']           = $name;
-		$arr['resource_type']   = 'nwikipage';
-		$arr['resource_id']     = $resource_id;
-		$arr['allow_cid']       = $w['wiki']['allow_cid'];
-		$arr['allow_gid']       = $w['wiki']['allow_gid'];
-		$arr['deny_cid']        = $w['wiki']['deny_cid'];
-		$arr['deny_gid']        = $w['wiki']['deny_gid'];
-		$arr['item_private']    = $w['wiki']['item_private'];
-		$arr['item_wall']       = 1;
-		$arr['item_origin']     = 1;
-		$arr['item_thread_top'] = 1;
-		$arr['verb']            = ACTIVITY_CREATE;
-		$arr['obj_type']        = 'Document';
-		// TODO: add an object?
-		$arr['public_policy'] = map_scope(PermissionLimits::Get($channel['channel_id'], 'view_wiki'), true);
-
-		// We may wish to change this some day.
-		$arr['item_unpublished'] = 1;
-
-		set_iconfig($arr, 'nwikipage', 'pagetitle', (($name) ? $name : t('(No Title)')), true);
-		$p = item_store($arr, false, false);
-
-		if ($p['item_id']) {
-			$page = [
-				'rawName'  => $name,
-				'htmlName' => escape_tags($name),
-				//'urlName'  => urlencode($name),
-				'urlName'  => NativeWiki::name_encode($name)
-
-			];
-
-			return ['page' => $page, 'item_id' => $p['item_id'], 'item' => $p['activity'], 'wiki' => $w, 'message' => '', 'success' => true];
-		}
-		return ['success' => false, 'message' => t('Wiki page create failed.')];
-	}
-
-
-	static public function rename_page($arr) {
-
-		$pageUrlName   = ((array_key_exists('pageUrlName', $arr)) ? $arr['pageUrlName'] : '');
-		$pageNewName   = ((array_key_exists('pageNewName', $arr)) ? $arr['pageNewName'] : '');
-		$resource_id   = ((array_key_exists('resource_id', $arr)) ? $arr['resource_id'] : '');
-		$observer_hash = ((array_key_exists('observer_hash', $arr)) ? $arr['observer_hash'] : '');
-		$channel_id    = ((array_key_exists('channel_id', $arr)) ? $arr['channel_id'] : 0);
-
-		$w = NativeWiki::get_wiki($channel_id, $observer_hash, $resource_id);
-		if (!$w['wiki']) {
-			return ['message' => t('Wiki not found.'), 'success' => false];
-		}
-
-
-		$ic = q("select * from iconfig left join item on iconfig.iid = item.id
-			where uid = %d and cat = 'nwikipage' and k = 'pagetitle' and v = '%s'",
-			intval($channel_id),
-			dbesc($pageNewName)
-		);
-
-		if ($ic) {
-			return ['success' => false, 'message' => t('Destination name already exists')];
-		}
-
-
-		$ids = [];
-
-		$ic = q("select *, item.id as item_id from iconfig left join item on iconfig.iid = item.id
-			where uid = %d and cat = 'nwikipage' and k = 'pagetitle' and v = '%s'",
-			intval($channel_id),
-			dbesc($pageUrlName)
-		);
-
-		if ($ic) {
-			foreach ($ic as $c) {
-				set_iconfig($c['item_id'], 'nwikipage', 'pagetitle', $pageNewName);
-				$ids[] = $c['item_id'];
-			}
-
-			$str_ids = implode(',', $ids);
-			q("update item set title = '%s' where id in ($str_ids)",
-				dbesc($pageNewName)
-			);
-
-			$page = [
-				'rawName'  => $pageNewName,
-				'htmlName' => escape_tags($pageNewName),
-				//'urlName'  => urlencode(escape_tags($pageNewName))
-				'urlName'  => NativeWiki::name_encode($pageNewName)
-			];
-
-			return ['success' => true, 'page' => $page];
-		}
-
-		return ['success' => false, 'message' => t('Page not found')];
-
-	}
-
-
-	static public function get_page_content($arr) {
-
-		$resource_id   = ((array_key_exists('resource_id', $arr)) ? $arr['resource_id'] : '');
-		$observer_hash = ((array_key_exists('observer_hash', $arr)) ? $arr['observer_hash'] : '');
-		$channel_id    = ((array_key_exists('channel_id', $arr)) ? intval($arr['channel_id']) : 0);
-
-		$w = NativeWiki::get_wiki($channel_id, $observer_hash, $resource_id);
-		if (!$w['wiki']) {
-			return ['content' => null, 'message' => 'Error reading wiki', 'success' => false];
-		}
-
-		$item = self::load_page($arr);
-
-		if ($item) {
-			$content = $item['body'];
-
-			return [
-				'content'      => $content,
-				'mimeType'     => $w['mimeType'],
-				'pageMimeType' => $item['mimetype'],
-				'message'      => '',
-				'success'      => true
-			];
-		}
-
-		return ['content' => null, 'message' => t('Error reading page content'), 'success' => false];
-
-	}
-
-
-	static public function page_history($arr) {
-
-		$resource_id   = ((array_key_exists('resource_id', $arr)) ? $arr['resource_id'] : '');
-		$observer_hash = ((array_key_exists('observer_hash', $arr)) ? $arr['observer_hash'] : '');
-		$channel_id    = ((array_key_exists('channel_id', $arr)) ? $arr['channel_id'] : 0);
-
-		$w = NativeWiki::get_wiki($channel_id, $observer_hash, $resource_id);
-		if (!$w['wiki']) {
-			return ['history' => null, 'message' => 'Error reading wiki', 'success' => false];
-		}
-
-		$items = self::load_page_history($arr);
-
-		$history = [];
-
-		if ($items) {
-			$processed = 0;
-			foreach ($items as $item) {
-				if ($processed > 1000)
-					break;
-				$processed++;
-				$history[] = [
-					'revision' => $item['revision'],
-					'date'     => datetime_convert('UTC', date_default_timezone_get(), $item['edited']),
-					'name'     => $item['author']['xchan_name'],
-					'title'    => get_iconfig($item, 'nwikipage', 'commit_msg')
-				];
-
-			}
-
-			return ['success' => true, 'history' => $history];
-		}
-
-		return ['success' => false];
-
-	}
-
-
-	static public function load_page($arr) {
-
-		$pageUrlName   = ((array_key_exists('pageUrlName', $arr)) ? $arr['pageUrlName'] : '');
-		$resource_id   = ((array_key_exists('resource_id', $arr)) ? $arr['resource_id'] : '');
-		$observer_hash = ((array_key_exists('observer_hash', $arr)) ? $arr['observer_hash'] : '');
-		$channel_id    = ((array_key_exists('channel_id', $arr)) ? $arr['channel_id'] : 0);
-		$revision      = ((array_key_exists('revision', $arr)) ? $arr['revision'] : (-1));
-
-		$w = NativeWiki::get_wiki($channel_id, $observer_hash, $resource_id);
-
-		if (!$w['wiki']) {
-			return ['content' => null, 'message' => 'Error reading wiki', 'success' => false];
-		}
-
-		$ids = '';
-
-		$ic = q("select * from iconfig left join item on iconfig.iid = item.id where uid = %d and cat = 'nwikipage' and k = 'pagetitle' and v = '%s'",
-			intval($channel_id),
-			dbesc($pageUrlName)
-		);
-
-		if ($ic) {
-			foreach ($ic as $c) {
-				if ($ids)
-					$ids .= ',';
-				$ids .= intval($c['iid']);
-			}
-		}
-
-		$sql_extra = item_permissions_sql($channel_id, $observer_hash);
-
-		if ($revision == (-1))
-			$sql_extra .= " order by revision desc ";
-		elseif ($revision)
-			$sql_extra .= " and revision = " . intval($revision) . " ";
-
-		$r = null;
-
-
-		if ($ids) {
-			$r = q("select * from item where resource_type = 'nwikipage' and resource_id = '%s' and uid = %d and id in ( $ids ) $sql_extra limit 1",
-				dbesc($resource_id),
-				intval($channel_id)
-			);
-
-			if ($r) {
-				$items = fetch_post_tags($r, true);
-				return $items[0];
-			}
-		}
-
-		return null;
-	}
-
-
-	static public function load_page_history($arr) {
-
-		$pageUrlName   = ((array_key_exists('pageUrlName', $arr)) ? $arr['pageUrlName'] : '');
-		$resource_id   = ((array_key_exists('resource_id', $arr)) ? $arr['resource_id'] : '');
-		$observer_hash = ((array_key_exists('observer_hash', $arr)) ? $arr['observer_hash'] : '');
-		$channel_id    = ((array_key_exists('channel_id', $arr)) ? $arr['channel_id'] : 0);
-
-		$w = NativeWiki::get_wiki($channel_id, $observer_hash, $resource_id);
-		if (!$w['wiki']) {
-			return ['content' => null, 'message' => 'Error reading wiki', 'success' => false];
-		}
-
-		$ids = '';
-
-		$ic = q("select * from iconfig left join item on iconfig.iid = item.id where uid = %d and cat = 'nwikipage' and k = 'pagetitle' and v = '%s'",
-			intval($channel_id),
-			dbesc($pageUrlName)
-		);
-
-		if ($ic) {
-			foreach ($ic as $c) {
-				if ($ids)
-					$ids .= ',';
-				$ids .= intval($c['iid']);
-			}
-		}
-
-		$sql_extra = item_permissions_sql($channel_id, $observer_hash);
-
-		$sql_extra .= " order by revision desc ";
-
-		$r = null;
-		if ($ids) {
-			$r = q("select * from item where resource_type = 'nwikipage' and resource_id = '%s' and uid = %d and id in ( $ids ) and item_deleted = 0 $sql_extra",
-				dbesc($resource_id),
-				intval($channel_id)
-			);
-			if ($r) {
-				xchan_query($r);
-				$items = fetch_post_tags($r, true);
-				return $items;
-			}
-		}
-
-		return null;
-	}
-
-
-	static public function save_page($arr) {
-		$pageUrlName   = ((array_key_exists('pageUrlName', $arr)) ? $arr['pageUrlName'] : '');
-		$content       = ((array_key_exists('content', $arr)) ? $arr['content'] : '');
-		$resource_id   = ((array_key_exists('resource_id', $arr)) ? $arr['resource_id'] : '');
-		$observer_hash = ((array_key_exists('observer_hash', $arr)) ? $arr['observer_hash'] : '');
-		$channel_id    = ((array_key_exists('channel_id', $arr)) ? $arr['channel_id'] : 0);
-
-		$w = NativeWiki::get_wiki($channel_id, $observer_hash, $resource_id);
-
-		if (!$w['wiki']) {
-			return ['message' => t('Error reading wiki'), 'success' => false];
-		}
-
-
-		// fetch the most recently saved revision.
-
-		$item = self::load_page($arr);
-
-		if (!$item) {
-			return ['message' => t('Page not found'), 'success' => false];
-		}
-
-		$mimetype = $item['mimetype'];
-
-		// change just the fields we need to change to create a revision;
-
-		unset($item['id']);
-		unset($item['author']);
-		$item['parent']       = 0;
-		$item['body']         = $content;
-		$item['author_xchan'] = $observer_hash;
-		$item['revision']     = (($arr['revision']) ? intval($arr['revision']) + 1 : intval($item['revision']) + 1);
-		$item['edited']       = datetime_convert();
-		$item['mimetype']     = $mimetype;
-
-		if ($item['iconfig'] && is_array($item['iconfig']) && count($item['iconfig'])) {
-			for ($x = 0; $x < count($item['iconfig']); $x++) {
-				unset($item['iconfig'][$x]['id']);
-				unset($item['iconfig'][$x]['iid']);
-			}
-		}
-
-		$ret = item_store($item, false, false);
-
-		if ($ret['item_id'])
-			return ['message' => '', 'item_id' => $ret['item_id'], 'filename' => $pageUrlName, 'success' => true];
-		else
-			return ['message' => t('Page update failed.'), 'success' => false];
-	}
-
-
-	static public function delete_page($arr) {
-
-		$pageUrlName   = (array_key_exists('pageUrlName', $arr) ? $arr['pageUrlName'] : '');
-		$resource_id   = (array_key_exists('resource_id', $arr) ? $arr['resource_id'] : '');
-		$observer_hash = (array_key_exists('observer_hash', $arr) ? $arr['observer_hash'] : '');
-		$channel_id    = (array_key_exists('channel_id', $arr) ? $arr['channel_id'] : 0);
-
-		$w = NativeWiki::get_wiki($channel_id, $observer_hash, $resource_id);
-		if (!$w['wiki']) {
-			return ['success' => false, 'message' => t('Error reading wiki')];
-		}
-
-		$ids = [];
-
-		$ic = q("select * from iconfig left join item on iconfig.iid = item.id
-			where uid = %d and cat = 'nwikipage' and k = 'pagetitle' and v = '%s'",
-			intval($channel_id),
-			dbesc($pageUrlName)
-		);
-
-		if ($ic) {
-			foreach ($ic as $c) {
-				$ids[] = intval($c['iid']);
-			}
-		}
-
-		if ($ids) {
-			drop_items($ids, true, DROPITEM_PHASE1);
-			return ['success' => true];
-		}
-
-		return ['success' => false, 'message' => t('Nothing deleted')];
-	}
-
-
-	static public function revert_page($arr) {
-
-		$resource_id   = ((array_key_exists('resource_id', $arr)) ? $arr['resource_id'] : '');
-		$commitHash    = ((array_key_exists('commitHash', $arr)) ? $arr['commitHash'] : null);
-		$observer_hash = ((array_key_exists('observer_hash', $arr)) ? $arr['observer_hash'] : '');
-		$channel_id    = ((array_key_exists('channel_id', $arr)) ? $arr['channel_id'] : 0);
-
-		if (!$commitHash) {
-			return ['message' => 'No commit was provided', 'success' => false];
-		}
-
-		$w = NativeWiki::get_wiki($channel_id, $observer_hash, $resource_id);
-		if (!$w['wiki']) {
-			return ['message' => 'Error reading wiki', 'success' => false];
-		}
-
-		$x = $arr;
-
-		if (intval($commitHash) > 0) {
-			unset($x['commitHash']);
-			$x['revision'] = intval($commitHash) - 1;
-			$loaded        = self::load_page($x);
-
-			if ($loaded) {
-				$content = $loaded['body'];
-				return ['content' => $content, 'success' => true];
-			}
-			return ['success' => false];
-		}
-	}
-
-
-	static public function compare_page($arr) {
-
-		$resource_id   = ((array_key_exists('resource_id', $arr)) ? $arr['resource_id'] : '');
-		$compareCommit = ((array_key_exists('compareCommit', $arr)) ? $arr['compareCommit'] : 0);
-		$observer_hash = ((array_key_exists('observer_hash', $arr)) ? $arr['observer_hash'] : '');
-		$channel_id    = ((array_key_exists('channel_id', $arr)) ? $arr['channel_id'] : 0);
-
-		$w = NativeWiki::get_wiki($channel_id, $observer_hash, $resource_id);
-
-		if (!$w['wiki']) {
-			return ['message' => t('Error reading wiki'), 'success' => false];
-		}
-
-		$x             = $arr;
-		$x['revision'] = (-1);
-
-		$currpage = self::load_page($x);
-		if ($currpage)
-			$currentContent = $currpage['body'];
-
-		$x['revision'] = $compareCommit;
-		$comppage      = self::load_page($x);
-		if ($comppage)
-			$compareContent = $comppage['body'];
-
-		if ($currpage && $comppage) {
-			require_once('library/class.Diff.php');
-			$diff = \Diff::toTable(\Diff::compare($currentContent, $compareContent));
-
-			return ['success' => true, 'diff' => $diff];
-		}
-		return ['success' => false, 'message' => t('Compare: object not found.')];
-
-	}
-
-
-	static public function commit($arr) {
-
-		$commit_msg    = ((array_key_exists('commit_msg', $arr)) ? $arr['commit_msg'] : t('Page updated'));
-		$observer_hash = ((array_key_exists('observer_hash', $arr)) ? $arr['observer_hash'] : '');
-		$channel_id    = ((array_key_exists('channel_id', $arr)) ? $arr['channel_id'] : 0);
-
-		if (array_key_exists('resource_id', $arr)) {
-			$resource_id = $arr['resource_id'];
-		}
-		else {
-			return ['message' => t('Wiki resource_id required for git commit'), 'success' => false];
-		}
-
-		$w = NativeWiki::get_wiki($channel_id, $observer_hash, $resource_id);
-		if (!$w['wiki']) {
-			return ['message' => t('Error reading wiki'), 'success' => false];
-		}
-
-
-		$page = self::load_page($arr);
-
-		if ($page) {
-			set_iconfig($page['id'], 'nwikipage', 'commit_msg', escape_tags($commit_msg), true);
-			return ['success' => true, 'item_id' => $page['id'], 'page' => $page];
-		}
-
-		return ['success' => false, 'message' => t('Page not found.')];
-
-	}
-
-	static public function convert_links($s, $wikiURL) {
-
-		if (strpos($s, '[[') !== false) {
-			preg_match_all("/\[\[(.*?)\]\]/", $s, $match);
-			$pages = $pageURLs = [];
-			foreach ($match[1] as $m) {
-				// TODO: Why do we need to double urlencode for this to work?
-				//$pageURLs[] = urlencode(urlencode(escape_tags($m)));
-				$titleUri   = explode('|', $m);
-				$page       = $titleUri[0] ?? '';
-				$title      = $titleUri[1] ?? $page;
-				$pageURLs[] = NativeWiki::name_encode(escape_tags($page));
-				$pages[]    = $title;
-			}
-			$idx = 0;
-			while (strpos($s, '[[') !== false) {
-				$replace = '<a href="' . $wikiURL . '/' . $pageURLs[$idx] . '">' . $pages[$idx] . '</a>';
-				$s       = preg_replace("/\[\[(.*?)\]\]/", $replace, $s, 1);
-				$idx++;
-			}
-		}
-		return $s;
-	}
-
-	static public function render_page_history($arr) {
-
-		$pageUrlName = ((array_key_exists('pageUrlName', $arr)) ? $arr['pageUrlName'] : '');
-		$resource_id = ((array_key_exists('resource_id', $arr)) ? $arr['resource_id'] : '');
-
-		$pageHistory = self::page_history([
-			'channel_id'    => App::$profile_uid,
-			'observer_hash' => get_observer_hash(),
-			'resource_id'   => $resource_id,
-			'pageUrlName'   => $pageUrlName
-		]);
-
-		return replace_macros(get_markup_template('nwiki_page_history.tpl'), [
-			'$pageHistory' => $pageHistory['history'],
-			'$permsWrite'  => $arr['permsWrite'],
-			'$name_lbl'    => t('Name'),
-			'$msg_label'   => t('Message', 'wiki_history'),
-			'$date_lbl'    => t('Date'),
-			'$revert_btn'  => t('Revert'),
-			'$compare_btn' => t('Compare')
-		]);
-
-	}
-
-
-	/**
-	 * Replace the instances of the string [toc] with a list element that will be populated by
-	 * a table of contents by the JavaScript library
-	 * @param string $s
-	 * @return string
-	 */
-	static public function generate_toc($s) {
-		if (strpos($s, '[toc]') !== false) {
-			//$toc_md = wiki_toc($s);	// Generate Markdown-formatted list prior to HTML render
-			$toc_md = '<ul id="wiki-toc"></ul>'; // use the available jQuery plugin http://ndabas.github.io/toc/
-			$s      = preg_replace("/\[toc\]/", $toc_md, $s, -1);
-		}
-		return $s;
-	}
-
-
-	/**
-	 *  Converts a select set of bbcode tags. Much of the code is copied from include/bbcode.php
-	 * @param string $s
-	 * @return string
-	 */
-	static public function bbcode($s) {
-
-		$s = str_replace(['[baseurl]', '[sitename]'], [z_root(), get_config('system', 'sitename')], $s);
-
-		$s = preg_replace_callback("/\[observer\.language\=(.*?)\](.*?)\[\/observer\]/ism", 'oblanguage_callback', $s);
-
-		$s = preg_replace_callback("/\[observer\.language\!\=(.*?)\](.*?)\[\/observer\]/ism", 'oblanguage_necallback', $s);
-
-
-		$observer = App::get_observer();
-		if ($observer) {
-			$s1         = '<span class="bb_observer" title="' . t('Different viewers will see this text differently') . '">';
-			$s2         = '</span>';
-			$obsBaseURL = $observer['xchan_connurl'];
-			$obsBaseURL = preg_replace("/\/poco\/.*$/", '', $obsBaseURL);
-			$s          = str_replace('[observer.baseurl]', $obsBaseURL, $s);
-			$s          = str_replace('[observer.url]', $observer['xchan_url'], $s);
-			$s          = str_replace('[observer.name]', $s1 . $observer['xchan_name'] . $s2, $s);
-			$s          = str_replace('[observer.address]', $s1 . $observer['xchan_addr'] . $s2, $s);
-			$s          = str_replace('[observer.webname]', substr($observer['xchan_addr'], 0, strpos($observer['xchan_addr'], '@')), $s);
-			$s          = str_replace('[observer.photo]', '', $s);
-		}
-		else {
-			$s = str_replace('[observer.baseurl]', '', $s);
-			$s = str_replace('[observer.url]', '', $s);
-			$s = str_replace('[observer.name]', '', $s);
-			$s = str_replace('[observer.address]', '', $s);
-			$s = str_replace('[observer.webname]', '', $s);
-			$s = str_replace('[observer.photo]', '', $s);
-		}
-
-		return $s;
-	}
-
-
-	static public function get_file_ext($arr) {
-
-		if ($arr['mimetype'] === 'text/bbcode')
-			return '.bb';
-		elseif ($arr['mimetype'] === 'text/markdown')
-			return '.md';
-		elseif ($arr['mimetype'] === 'text/plain')
-			return '.txt';
-
-	}
-
-	// This function is derived from
-	// http://stackoverflow.com/questions/32068537/generate-table-of-contents-from-markdown-in-php
-	static public function toc($content) {
-		// ensure using only "\n" as line-break
-		$source = str_replace(["\r\n", "\r"], "\n", $content);
-
-		// look for markdown TOC items
-		preg_match_all(
-			'/^(?:=|-|#).*$/m',
-			$source,
-			$matches,
-			PREG_PATTERN_ORDER | PREG_OFFSET_CAPTURE
-		);
-
-		// preprocess: iterate matched lines to create an array of items
-		// where each item is an array(level, text)
-		$file_size = strlen($source);
-		foreach ($matches[0] as $item) {
-			$found_mark = substr($item[0], 0, 1);
-			if ($found_mark == '#') {
-				// text is the found item
-				$item_text  = $item[0];
-				$item_level = strrpos($item_text, '#') + 1;
-				$item_text  = substr($item_text, $item_level);
-			}
-			else {
-				// text is the previous line (empty if <hr>)
-				$item_offset      = $item[1];
-				$prev_line_offset = strrpos($source, "\n", -($file_size - $item_offset + 2));
-				$item_text        =
-					substr($source, $prev_line_offset, $item_offset - $prev_line_offset - 1);
-				$item_text        = trim($item_text);
-				$item_level       = $found_mark == '=' ? 1 : 2;
-			}
-			if (!trim($item_text) or strpos($item_text, '|') !== FALSE) {
-				// item is an horizontal separator or a table header, don't mind
-				continue;
-			}
-			$raw_toc[] = ['level' => $item_level, 'text' => trim($item_text)];
-		}
-		$o = '';
-		foreach ($raw_toc as $t) {
-			$level = intval($t['level']);
-			$text  = $t['text'];
-			switch ($level) {
-				case 1:
-					$li = '* ';
-					break;
-				case 2:
-					$li = '  * ';
-					break;
-				case 3:
-					$li = '    * ';
-					break;
-				case 4:
-					$li = '      * ';
-					break;
-				default:
-					$li = '* ';
-					break;
-			}
-			$o .= $li . $text . "\n";
-		}
-		return $o;
-	}
-
-}

Zotlabs/Lib/Queue.php

@@ -57,7 +57,6 @@ class Queue {
 			outq_priority = outq_priority + %d,
 			outq_scheduled = '%s'
 			WHERE outq_hash = '%s'",
-
 			dbesc(datetime_convert()),
 			intval($add_priority),
 			dbesc($next),
@@ -65,16 +64,38 @@ class Queue {
 		);
 	}
 
-
-	static function remove($id,$channel_id = 0) {
-		logger('queue: remove queue item ' . $id,LOGGER_DEBUG);
+	public static function remove($id, $channel_id = 0) {
+		logger('queue: remove queue item ' . $id, LOGGER_DEBUG);
 		$sql_extra = (($channel_id) ? " and outq_channel = " . intval($channel_id) . " " : '');
 
-		q("DELETE FROM outq WHERE outq_hash = '%s' $sql_extra",
+		// figure out what endpoint it is going to.
+		$record = q("select outq_posturl from outq where outq_hash = '%s' $sql_extra",
 			dbesc($id)
 		);
-	}
 
+		if ($record) {
+			q("DELETE FROM outq WHERE outq_hash = '%s' $sql_extra",
+				dbesc($id)
+			);
+
+			// If there's anything remaining in the queue for this site, move one of them to the next active
+			// queue run by setting outq_scheduled back to the present. We may be attempting to deliver it
+			// as a 'piled_up' delivery, but this ensures the site has an active queue entry as long as queued
+			// entries still exist for it. This fixes an issue where one immediate delivery left everything
+			// else for that site undeliverable since all the other entries had been pushed far into the future.
+
+			$r = q("SELECT outq_hash, outq_posturl FROM outq WHERE outq_posturl = '%s' LIMIT 1",
+				dbesc($record[0]['outq_posturl'])
+			);
+
+			if ($r) {
+				$hashes = ids_to_querystr($r, 'outq_hash', true);
+				$x = q("UPDATE outq SET outq_scheduled = '%s' WHERE outq_hash IN ($hashes)",
+					dbesc(datetime_convert())
+				);
+			}
+		}
+	}
 
 	static function remove_by_posturl($posturl) {
 		logger('queue: remove queue posturl ' . $posturl,LOGGER_DEBUG);
@@ -84,8 +105,6 @@ class Queue {
 		);
 	}
 
-
-
 	static function set_delivered($id,$channel = 0) {
 		logger('queue: set delivered ' . $id,LOGGER_DEBUG);
 		$sql_extra = (($channel['channel_id']) ? " and outq_channel = " . intval($channel['channel_id']) . " " : '');
@@ -110,21 +129,30 @@ class Queue {
 			return false;
 		}
 
+		$hash = $arr['hash'] ?? '';
+		$account_id = $arr['account_id'] ?? 0;
+		$channel_id = $arr['channel_id'] ?? 0;
+		$driver = $arr['driver'] ?? 'zot6';
+		$posturl = $arr['posturl'] ?? '';
+		$priority = $arr['priority'] ?? 0;
+		$notify = $arr['notify'] ?? '';
+		$msg = $arr['msg'] ?? '';
+
 		$x = q("insert into outq ( outq_hash, outq_account, outq_channel, outq_driver, outq_posturl, outq_async, outq_priority,
 			outq_created, outq_updated, outq_scheduled, outq_notify, outq_msg )
 			values ( '%s', %d, %d, '%s', '%s', %d, %d, '%s', '%s', '%s', '%s', '%s' )",
-			dbesc($arr['hash']),
-			intval($arr['account_id']),
-			intval($arr['channel_id']),
-			dbesc(($arr['driver']) ? $arr['driver'] : 'zot6'),
-			dbesc($arr['posturl']),
+			dbesc($hash),
+			intval($account_id),
+			intval($channel_id),
+			dbesc($driver),
+			dbesc($posturl),
 			intval(1),
-			intval(isset($arr['priority']) ? $arr['priority'] : 0),
+			intval($priority),
 			dbesc(datetime_convert()),
 			dbesc(datetime_convert()),
 			dbesc(datetime_convert()),
-			dbesc($arr['notify']),
-			dbesc(($arr['msg']) ? $arr['msg'] : '')
+			dbesc($notify),
+			dbesc($msg)
 		);
 		return $x;
 
@@ -143,17 +171,19 @@ class Queue {
 			$y = q("select site_update, site_dead from site where site_url = '%s' ",
 				dbesc($base)
 			);
-			if($y) {
-				if(intval($y[0]['site_dead'])) {
+
+			if ($y) {
+				// Don't bother delivering if the site is dead.
+				// And if we haven't heard from the site in over a month - let them through but 3 strikes you're out.
+				if (intval($y[0]['site_dead']) || ($y[0]['site_update'] < datetime_convert('UTC', 'UTC', 'now - 1 month') && $outq['outq_priority'] > 20)) {
+					q("update dreport set dreport_result = '%s' where dreport_queue = '%s'",
+						dbesc('site dead'),
+						dbesc($outq['outq_hash'])
+					);
 					self::remove_by_posturl($outq['outq_posturl']);
 					logger('dead site ignored ' . $base);
 					return;
 				}
-				if($y[0]['site_update'] < datetime_convert('UTC','UTC','now - 1 month')) {
-					self::update($outq['outq_hash'], 10);
-					logger('immediate delivery deferred for site ' . $base);
-					return;
-				}
 			}
 			else {
 
@@ -202,7 +232,7 @@ class Queue {
 
 			if($channel && $base) {
 				$h = q("SELECT hubloc_sitekey, site_crypto FROM hubloc LEFT JOIN site ON hubloc_url = site_url
-					WHERE site_url = '%s' AND hubloc_network = 'zot6' ORDER BY hubloc_id DESC LIMIT 1",
+					WHERE site_url = '%s' AND hubloc_network = 'zot6' AND hubloc_deleted = 0 ORDER BY hubloc_primary DESC, hubloc_id DESC LIMIT 1",
 					dbesc($base)
 				);
 				if($h) {
@@ -216,7 +246,7 @@ class Queue {
 
 			if($result['success']) {
 				logger('deliver: remote zot delivery succeeded to ' . $outq['outq_posturl']);
-				Libzot::process_response($outq['outq_posturl'],$result, $outq);
+				Libzot::process_response($outq['outq_posturl'], $result, $outq);
 			}
 			else {
 				logger('deliver: remote zot delivery failed to ' . $outq['outq_posturl']);

Zotlabs/Lib/QueueWorker.php

@@ -0,0 +1,384 @@
+<?php
+
+namespace Zotlabs\Lib;
+
+use Ramsey\Uuid\Uuid;
+use Ramsey\Uuid\Exception\UnableToBuildUuidException;
+
+class QueueWorker {
+
+	public static $queueworker = null;
+	public static $maxworkers = 0;
+	public static $workermaxage = 0;
+	public static $workersleep = 100;
+	public static $default_priorities = [
+		'Notifier'         => 10,
+		'Deliver'          => 10,
+		'Cache_query'      => 10,
+		'Content_importer' => 1,
+		'File_importer'    => 1,
+		'Channel_purge'    => 1,
+		'Directory'        => 1
+	];
+
+	// Exceptions for processtimeout ($workermaxage) value.
+	// Currently the value is overriden with 3600 seconds (1h).
+	public static $long_running_cmd = [
+		'Queue',
+		'Expire'
+	];
+
+	private static function qstart() {
+		q('START TRANSACTION');
+	}
+
+	private static function qcommit() {
+		q("COMMIT");
+	}
+
+	private static function qrollback() {
+		q("ROLLBACK");
+	}
+
+	public static function Summon($argv) {
+
+		if ($argv[0] !== 'Queueworker') {
+
+			$priority = 0; // @TODO allow reprioritization
+
+			if (isset(self::$default_priorities[$argv[0]])) {
+				$priority = self::$default_priorities[$argv[0]];
+			}
+
+			$workinfo      = ['argc' => count($argv), 'argv' => $argv];
+			$workinfo_json = json_encode($workinfo);
+			$uuid          = self::getUuid($workinfo_json);
+
+			$r = q("SELECT * FROM workerq WHERE workerq_uuid = '%s'",
+				dbesc($uuid)
+			);
+			if ($r) {
+				logger("Summon: Ignoring duplicate workerq task", LOGGER_DEBUG);
+				logger(print_r($workinfo, true));
+				return;
+			}
+
+			logger('queueworker_stats_summon: cmd:' . $argv[0] . ' ' . 'timestamp:' . time());
+
+			self::qstart();
+			$r = q("INSERT INTO workerq (workerq_priority, workerq_data, workerq_uuid, workerq_cmd) VALUES (%d, '%s', '%s', '%s')",
+				intval($priority),
+				$workinfo_json,
+				dbesc($uuid),
+				dbesc($argv[0])
+			);
+			if (!$r) {
+				self::qrollback();
+				logger("INSERT FAILED", LOGGER_DEBUG);
+				return;
+			}
+			self::qcommit();
+			logger('INSERTED: ' . $workinfo_json, LOGGER_DEBUG);
+		}
+
+		$workers = self::GetWorkerCount();
+		if ($workers < self::$maxworkers) {
+			logger($workers . '/' . self::$maxworkers . ' workers active', LOGGER_DEBUG);
+			$phpbin = get_config('system', 'phpbin', 'php');
+			proc_run($phpbin, 'Zotlabs/Daemon/Master.php', ['Queueworker']);
+		}
+	}
+
+	public static function Release($argv) {
+
+		if ($argv[0] !== 'Queueworker') {
+
+			$priority = 0; // @TODO allow reprioritization
+			if (isset(self::$default_priorities[$argv[0]])) {
+				$priority = self::$default_priorities[$argv[0]];
+			}
+
+			$workinfo      = ['argc' => count($argv), 'argv' => $argv];
+			$workinfo_json = json_encode($workinfo);
+			$uuid          = self::getUuid($workinfo_json);
+
+			$r = q("SELECT * FROM workerq WHERE workerq_uuid = '%s'",
+				dbesc($uuid)
+			);
+			if ($r) {
+				logger("Release: Duplicate task - do not insert.", LOGGER_DEBUG);
+				logger(print_r($workinfo, true));
+				return;
+			}
+
+			self::qstart();
+			$r = q("INSERT INTO workerq (workerq_priority, workerq_data, workerq_uuid, workerq_cmd) VALUES (%d, '%s', '%s', '%s')",
+				intval($priority),
+				$workinfo_json,
+				dbesc($uuid),
+				dbesc($argv[0])
+			);
+			if (!$r) {
+				self::qrollback();
+				logger("Insert failed: " . $workinfo_json, LOGGER_DEBUG);
+				return;
+			}
+			self::qcommit();
+			logger('INSERTED: ' . $workinfo_json, LOGGER_DEBUG);
+		}
+
+		self::Process();
+	}
+
+	public static function GetWorkerCount() {
+		if (self::$maxworkers == 0) {
+			self::$maxworkers = get_config('queueworker', 'max_queueworkers', 4);
+			self::$maxworkers = self::$maxworkers > 3 ? self::$maxworkers : 4;
+		}
+		if (self::$workermaxage == 0) {
+			self::$workermaxage = get_config('queueworker', 'max_queueworker_age');
+			self::$workermaxage = self::$workermaxage > 120 ? self::$workermaxage : 300;
+		}
+
+		self::qstart();
+
+		// skip locked is preferred but is not supported by mariadb < 10.6 which is still used a lot - hence make it optional
+		$sql_quirks = ((get_config('system', 'db_skip_locked_supported')) ? 'SKIP LOCKED' : 'NOWAIT');
+
+		$r = q("SELECT workerq_id FROM workerq WHERE workerq_reservationid IS NOT NULL AND workerq_processtimeout < %s FOR UPDATE $sql_quirks",
+			db_utcnow()
+		);
+
+		if ($r) {
+			// TODO: some long running services store their pid in config.procid.daemon
+			// we could possibly check if a pid exist and check if the process is still alive
+			// prior to reseting workerq_reservationid
+
+			$ids = ids_to_querystr($r, 'workerq_id');
+			$u = dbq("update workerq set workerq_reservationid = null where workerq_id in ($ids)");
+		}
+
+		self::qcommit();
+
+		//q("update workerq set workerq_reservationid = null where workerq_reservationid is not null and workerq_processtimeout < %s",
+			//db_utcnow()
+		//);
+
+		//usleep(self::$workersleep);
+
+		$workers = dbq("select count(*) as total from workerq where workerq_reservationid is not null");
+		logger("WORKERCOUNT: " . $workers[0]['total'], LOGGER_DEBUG);
+
+		return intval($workers[0]['total']);
+	}
+
+	public static function GetWorkerID() {
+		if (self::$queueworker) {
+			return self::$queueworker;
+		}
+
+		$wid = uniqid('', true);
+
+		//usleep(mt_rand(300000, 1000000)); //Sleep .3 - 1 seconds before creating a new worker.
+
+		$workers = self::GetWorkerCount();
+
+		if ($workers >= self::$maxworkers) {
+			logger("Too many active workers ($workers) max = " . self::$maxworkers, LOGGER_DEBUG);
+			return false;
+		}
+
+		self::$queueworker = $wid;
+
+		return $wid;
+	}
+
+	private static function getWorkId() {
+		self::GetWorkerCount();
+
+		self::qstart();
+
+		// skip locked is preferred but is not supported by mariadb < 10.6 which is still used a lot - hence make it optional
+		$sql_quirks = ((get_config('system', 'db_skip_locked_supported')) ? 'SKIP LOCKED' : 'NOWAIT');
+
+		$work = dbq("SELECT workerq_id, workerq_cmd FROM workerq WHERE workerq_reservationid IS NULL ORDER BY workerq_priority DESC, workerq_id ASC LIMIT 1 FOR UPDATE $sql_quirks");
+
+		if (!$work) {
+			self::qrollback();
+			return false;
+		}
+
+		$id = $work[0]['workerq_id'];
+		$cmd = $work[0]['workerq_cmd'];
+		$age = self::$workermaxage;
+
+		if (in_array($cmd, self::$long_running_cmd)) {
+			$age = 3600; // 1h TODO: make this configurable
+		}
+
+		$work = q("UPDATE workerq SET workerq_reservationid = '%s', workerq_processtimeout = %s + INTERVAL %s WHERE workerq_id = %d",
+			self::$queueworker,
+			db_utcnow(),
+			db_quoteinterval($age . " SECOND"),
+			intval($id)
+		);
+
+		if (!$work) {
+			self::qrollback();
+			logger("Could not update workerq.", LOGGER_DEBUG);
+			return false;
+		}
+
+		logger("GOTWORK: " . json_encode($work), LOGGER_DEBUG);
+		self::qcommit();
+
+		return $id;
+	}
+
+	public static function Process() {
+		$sleep = intval(get_config('queueworker', 'queue_worker_sleep', 100));
+		$auto_queue_worker_sleep = get_config('queueworker', 'auto_queue_worker_sleep', 0);
+
+		if (!self::GetWorkerID()) {
+			if ($auto_queue_worker_sleep) {
+				set_config('queueworker', 'queue_worker_sleep', $sleep + 100);
+			}
+
+			logger('Unable to get worker ID. Exiting.', LOGGER_DEBUG);
+			killme();
+		}
+
+		if ($auto_queue_worker_sleep && $sleep > 100) {
+			$next_sleep = $sleep - 100;
+			set_config('queueworker', 'queue_worker_sleep', (($next_sleep < 100) ? 100 : $next_sleep));
+		}
+
+		$jobs               = 0;
+		$workid             = self::getWorkId();
+		$load_average_sleep = false;
+		self::$workersleep  = $sleep;
+		self::$workersleep  = ((intval(self::$workersleep) > 100) ? intval(self::$workersleep) : 100);
+
+		if (function_exists('sys_getloadavg') && get_config('queueworker', 'load_average_sleep')) {
+			// very experimental!
+			$load_average_sleep = true;
+		}
+
+		while ($workid) {
+
+			if ($load_average_sleep) {
+				$load_average      = sys_getloadavg();
+				self::$workersleep = intval($load_average[0]) * 10000;
+
+				if (!self::$workersleep) {
+					self::$workersleep = 100;
+				}
+			}
+
+			logger('queue_worker_sleep: ' . self::$workersleep, LOGGER_DEBUG);
+
+			usleep(self::$workersleep);
+
+			$workitem = dbq("SELECT * FROM workerq WHERE workerq_id = $workid");
+
+			if ($workitem) {
+				// At least SOME work to do.... in case there's more, let's ramp up workers.
+				$workers = self::GetWorkerCount();
+
+				if ($workers < self::$maxworkers) {
+					logger($workers . '/' . self::$maxworkers . ' workers active', LOGGER_DEBUG);
+					$phpbin = get_config('system', 'phpbin', 'php');
+					proc_run($phpbin, 'Zotlabs/Daemon/Master.php', ['Queueworker']);
+				}
+
+				$jobs++;
+
+				logger("Workinfo: " . $workitem[0]['workerq_data'], LOGGER_DEBUG);
+
+				$workinfo = json_decode($workitem[0]['workerq_data'], true);
+				$argv     = $workinfo['argv'];
+
+				$cls  = '\\Zotlabs\\Daemon\\' . $argv[0];
+				$argv = flatten_array_recursive($argv);
+				$argc = count($argv);
+				$rnd = random_string(16);
+
+				logger('PROCESSING: ' . $rnd . ' ' . print_r($argv[0], true));
+
+				$start_timestamp = microtime(true);
+
+				$cls::run($argc, $argv);
+
+				logger('logger_stats_data cmd:' . $argv[0] . ' start:' . $start_timestamp . ' ' . 'end:' . microtime(true) . ' meta:' . $rnd);
+
+				logger('COMPLETED: ' . $rnd);
+
+				// @FIXME: Right now we assume that if we get a return, everything is OK.
+				// At some point we may want to test whether the run returns true/false
+				// and requeue the work to be tried again if needed.  But we probably want
+				// to implement some sort of "retry interval" first.
+
+				dbq("delete from workerq where workerq_id = $workid");
+			}
+			else {
+				logger("NO WORKITEM!", LOGGER_DEBUG);
+			}
+			$workid = self::getWorkId();
+		}
+		logger('Master: Worker Thread: queue items processed:' . $jobs, LOGGER_DEBUG);
+	}
+
+	public static function ClearQueue() {
+		$work = q("select * from workerq");
+		while ($work) {
+			foreach ($work as $workitem) {
+				$workinfo = json_decode($workitem['v'], true);
+				$argc     = $workinfo['argc'];
+				$argv     = $workinfo['argv'];
+
+				logger('Master: process: ' . print_r($argv, true), LOGGER_ALL, LOG_DEBUG);
+
+				if (!isset($argv[0])) {
+					q("delete from workerq where workerq_id = %d",
+						$work[0]['workerq_id']
+					);
+					continue;
+				}
+
+				$cls = '\\Zotlabs\\Daemon\\' . $argv[0];
+				$cls::run($argc, $argv);
+
+				q("delete from workerq where workerq_id = %d",
+					$work[0]['workerq_id']
+				);
+
+				//Give the server .3 seconds to catch its breath between tasks.
+				//This will hopefully keep it from crashing to it's knees entirely
+				//if the last task ended up initiating other parallel processes
+				//(eg. polling remotes)
+				usleep(300000);
+			}
+
+			//Make sure nothing new came in
+			$work = q("select * from workerq");
+		}
+	}
+
+	/**
+	 * @brief Generate a name-based v5 UUID with custom namespace
+	 *
+	 * @param string $data
+	 * @return string $uuid
+	 */
+	private static function getUuid(string $data) {
+		$namespace = '3a112e42-f147-4ccf-a78b-f6841339ea2a';
+		try {
+			$uuid = Uuid::uuid5($namespace, $data)->toString();
+		} catch (UnableToBuildUuidException $e) {
+			logger('UUID generation failed');
+			return '';
+		}
+		return $uuid;
+	}
+
+}

Zotlabs/Lib/System.php

@@ -16,13 +16,13 @@ class System {
 	}
 
 	static public function get_site_name() {
-		if(is_array(\App::$config) && is_array(\App::$config['system']) && \App::$config['system']['sitename'])
+		if(is_array(\App::$config) && is_array(\App::$config['system']) && isset(\App::$config['system']['sitename']))
 			return \App::$config['system']['sitename'];
 		return '';
 	}
 
 	static public function get_project_version() {
-		if(is_array(\App::$config) && is_array(\App::$config['system']) && \App::$config['system']['hide_version'])
+		if(is_array(\App::$config) && is_array(\App::$config['system']) && isset(\App::$config['system']['hide_version']))
 			return '';
 		if(is_array(\App::$config) && is_array(\App::$config['system']) && array_key_exists('std_version',\App::$config['system']))
 			return \App::$config['system']['std_version'];
@@ -31,33 +31,33 @@ class System {
 	}
 
 	static public function get_update_version() {
-		if(is_array(\App::$config) && is_array(\App::$config['system']) && \App::$config['system']['hide_version'])
+		if(is_array(\App::$config) && is_array(\App::$config['system']) && isset(\App::$config['system']['hide_version']))
 			return '';
 		return DB_UPDATE_VERSION;
 	}
 
 
 	static public function get_notify_icon() {
-		if(is_array(\App::$config) && is_array(\App::$config['system']) && \App::$config['system']['email_notify_icon_url'])
+		if(is_array(\App::$config) && is_array(\App::$config['system']) && isset(\App::$config['system']['email_notify_icon_url']))
 			return \App::$config['system']['email_notify_icon_url'];
 		return z_root() . DEFAULT_NOTIFY_ICON;
 	}
 
 	static public function get_site_icon() {
-		if(is_array(\App::$config) && is_array(\App::$config['system']) && \App::$config['system']['site_icon_url'])
+		if(is_array(\App::$config) && is_array(\App::$config['system']) && isset(\App::$config['system']['site_icon_url']))
 			return \App::$config['system']['site_icon_url'];
 		return z_root() . DEFAULT_PLATFORM_ICON ;
 	}
 
 
 	static public function get_project_link() {
-		if(is_array(\App::$config) && is_array(\App::$config['system']) && \App::$config['system']['project_link'])
+		if(is_array(\App::$config) && is_array(\App::$config['system']) && isset(\App::$config['system']['project_link']))
 			return \App::$config['system']['project_link'];
 		return 'https://hubzilla.org';
 	}
 
 	static public function get_project_srclink() {
-		if(is_array(\App::$config) && is_array(\App::$config['system']) && \App::$config['system']['project_srclink'])
+		if(is_array(\App::$config) && is_array(\App::$config['system']) && isset(\App::$config['system']['project_srclink']))
 			return \App::$config['system']['project_srclink'];
 		return 'https://framagit.org/hubzilla/core.git';
 	}
@@ -68,7 +68,7 @@ class System {
 
 
 	static public function get_zot_revision() {
-		$x = [ 'revision' => ZOT_REVISION ]; 
+		$x = [ 'revision' => ZOT_REVISION ];
 		call_hooks('zot_revision',$x);
 		return $x['revision'];
 	}

Zotlabs/Lib/ThreadItem.php

@@ -84,10 +84,8 @@ class ThreadItem {
 
 	public function get_template_data($conv_responses, $thread_level=1, $conv_flags = []) {
 
-		$result = array();
-
-		$item     = $this->get_data();
-
+		$result = [];
+		$item = $this->get_data();
 		$commentww = '';
 		$sparkle = '';
 		$buttons = '';
@@ -98,12 +96,12 @@ class ThreadItem {
 		$is_item = false;
 		$osparkle = '';
 		$total_children = $this->count_descendants();
-		$unseen_comments = (($item['real_uid']) ? 0 : $this->count_unseen_descendants());
+		$unseen_comments = ((isset($item['real_uid']) && $item['real_uid']) ? 0 : $this->count_unseen_descendants());
 
  		$conv = $this->get_conversation();
 		$observer = $conv->get_observer();
 
-		$acl = new AccessList(false);
+		$acl = new AccessList([]);
 		$acl->set($item);
 
 		$lock = ((intval($item['item_private']) || ($item['uid'] == local_channel() && $acl->is_private()))
@@ -139,16 +137,10 @@ class ThreadItem {
 			$shareable = false;
 		}
 
-		$privacy_warning = false;
-		if(intval($item['item_private']) && ($item['owner']['xchan_network'] === 'activitypub')) {
-			$recips = get_iconfig($item['parent'], 'activitypub', 'recips');
-
-			if(! is_array($recips['to']) || ! in_array($observer['xchan_url'], $recips['to']))
-				$privacy_warning = true;
-		}
+		$privacy_warning = ($item['owner']['xchan_network'] === 'activitypub' && intval($item['item_private']) === 1);
 
 		if ($lock) {
- 			if (($item['mid'] == $item['parent_mid']) && count(get_terms_oftype($item['term'],TERM_FORUM))) {
+ 			if (($item['mid'] == $item['parent_mid']) && isset($item['term']) && count(get_terms_oftype($item['term'], TERM_FORUM))) {
  				$privacy_warning = true;
 				$conv_flags['parent_privacy_warning'] = true;
  			}
@@ -162,24 +154,11 @@ class ThreadItem {
 
 		$mode = $conv->get_mode();
 
-		switch($item['item_type']) {
-			case ITEM_TYPE_CARD:
-				$edlink = 'card_edit';
-				break;
-			case ITEM_TYPE_ARTICLE:
-				$edlink = 'article_edit';
-				break;
-			default:
-				$edlink = 'editpost';
-				break;
-		}
-
 		if(local_channel() && $observer['xchan_hash'] === $item['author_xchan'])
-			$edpost = array(z_root() . '/' . $edlink . '/' . $item['id'], t('Edit'));
+			$edpost = array(z_root() . '/editpost/' . $item['id'], t('Edit'));
 		else
 			$edpost = false;
 
-
 		if($observer && $observer['xchan_hash']
 			&& ($observer['xchan_hash'] == $this->get_data_value('author_xchan')
 			|| $observer['xchan_hash'] == $this->get_data_value('owner_xchan')
@@ -193,7 +172,7 @@ class ThreadItem {
 			$dropping = false;
 		}
 
-
+		$drop = [];
 		if($dropping) {
 			$drop = array(
 				'dropping' => $dropping,
@@ -204,29 +183,21 @@ class ThreadItem {
 			$drop = [ 'dropping' => true, 'delete' => t('Admin Delete') ];
 		}
 
-// FIXME
-		if($observer_is_pageowner) {
-			$multidrop = array(
-				'select' => t('Select'),
-			);
-		}
-
 		$filer = ((($conv->get_profile_owner() == local_channel()) && (! array_key_exists('real_uid',$item))) ? t("Save to Folder") : false);
 
-		$profile_avatar = $item['author']['xchan_photo_m'];
+		$profile_avatar = $item['author']['xchan_photo_s'];
 		$profile_link   = chanlink_hash($item['author_xchan']);
 		$profile_name   = $item['author']['xchan_name'];
 
 		$location = format_location($item);
 		$isevent = false;
 		$attend = null;
-		$canvote = false;
 
 		// process action responses - e.g. like/dislike/attend/agree/whatever
 		$response_verbs = array('like');
 		if(feature_enabled($conv->get_profile_owner(),'dislike'))
 			$response_verbs[] = 'dislike';
-		if($item['obj_type'] === ACTIVITY_OBJ_EVENT) {
+		if(in_array($item['obj_type'], ['Event', ACTIVITY_OBJ_EVENT])) {
 			$response_verbs[] = 'attendyes';
 			$response_verbs[] = 'attendno';
 			$response_verbs[] = 'attendmaybe';
@@ -240,17 +211,6 @@ class ThreadItem {
 			$response_verbs[] = 'answer';
 		}
 
-		$consensus = (intval($item['item_consensus']) ? true : false);
-		if($consensus) {
-			$response_verbs[] = 'agree';
-			$response_verbs[] = 'disagree';
-			$response_verbs[] = 'abstain';
-			if($this->is_commentable() && $observer) {
-				$conlabels = array( t('I agree'), t('I disagree'), t('I abstain'));
-				$canvote = true;
-			}
-		}
-
 		if(! feature_enabled($conv->get_profile_owner(),'dislike'))
 			unset($conv_responses['dislike']);
 
@@ -258,7 +218,8 @@ class ThreadItem {
 
 		$my_responses = [];
 		foreach($response_verbs as $v) {
-			$my_responses[$v] = (($conv_responses[$v][$item['mid'] . '-m']) ? 1 : 0);
+
+			$my_responses[$v] = ((isset($conv_responses[$v][$item['mid'] . '-m'])) ? 1 : 0);
 		}
 
 		$like_count = ((x($conv_responses['like'],$item['mid'])) ? $conv_responses['like'][$item['mid']] : '');
@@ -271,6 +232,7 @@ class ThreadItem {
 		}
 		$like_button_label = tt('Like','Likes',$like_count,'noun');
 
+		$showdislike = '';
 		if (feature_enabled($conv->get_profile_owner(),'dislike')) {
 			$dislike_count = ((x($conv_responses['dislike'],$item['mid'])) ? $conv_responses['dislike'][$item['mid']] : '');
 			$dislike_list = ((x($conv_responses['dislike'],$item['mid'])) ? $conv_responses['dislike'][$item['mid'] . '-l'] : '');
@@ -281,11 +243,11 @@ class ThreadItem {
 			} else {
 				$dislike_list_part = '';
 			}
+
+			$showdislike = ((x($conv_responses['dislike'],$item['mid'])) ? format_like($conv_responses['dislike'][$item['mid']],$conv_responses['dislike'][$item['mid'] . '-l'],'dislike',$item['mid']) : '');
 		}
 
 		$showlike    = ((x($conv_responses['like'],$item['mid'])) ? format_like($conv_responses['like'][$item['mid']],$conv_responses['like'][$item['mid'] . '-l'],'like',$item['mid']) : '');
-		$showdislike = ((x($conv_responses['dislike'],$item['mid']) && feature_enabled($conv->get_profile_owner(),'dislike'))
-				? format_like($conv_responses['dislike'][$item['mid']],$conv_responses['dislike'][$item['mid'] . '-l'],'dislike',$item['mid']) : '');
 
 		/*
 		 * We should avoid doing this all the time, but it depends on the conversation mode
@@ -296,14 +258,11 @@ class ThreadItem {
 		$this->check_wall_to_wall();
 
 		if($this->is_toplevel()) {
-			// FIXME check this permission
-			if(($conv->get_profile_owner() == local_channel()) && (! array_key_exists('real_uid',$item))) {
-
-				$star = array(
+			if((local_channel() && $conv->get_profile_owner() === local_channel()) || (local_channel() && App::$module === 'pubstream')) {
+				$star = [
 					'toggle' => t("Toggle Star Status"),
 					'isstarred' => ((intval($item['item_starred'])) ? true : false),
-				);
-
+				];
 			}
 		}
 		else {
@@ -320,7 +279,7 @@ class ThreadItem {
 		$tagger = [];
 
 		// FIXME - check this permission
-		if($conv->get_profile_owner() == local_channel()) {
+		if(local_channel() && $conv->get_profile_owner() == local_channel()) {
 			/* disable until we agree on how to implemnt this in zot6/activitypub
 			$tagger = array(
 				'tagit' => t("Add Tag"),
@@ -332,7 +291,7 @@ class ThreadItem {
 		}
 
 		$has_bookmarks = false;
-		if(Apps::system_app_installed(local_channel(), 'Bookmarks') && is_array($item['term'])) {
+		if(Apps::system_app_installed(local_channel(), 'Bookmarks') && isset($item['term']) && is_array($item['term'])) {
 			foreach($item['term'] as $t) {
 				if(($t['ttype'] == TERM_BOOKMARK))
 					$has_bookmarks = true;
@@ -340,23 +299,29 @@ class ThreadItem {
 		}
 
 		$has_event = false;
-		if(($item['obj_type'] === ACTIVITY_OBJ_EVENT) && $conv->get_profile_owner() == local_channel())
+		if((in_array($item['obj_type'], ['Event', ACTIVITY_OBJ_EVENT])) && $conv->get_profile_owner() == local_channel())
 			$has_event = true;
 
+		$like = [];
+		$dislike = [];
+		$reply_to = [];
+
 		if($this->is_commentable() && $observer) {
 			$like = array( t("I like this \x28toggle\x29"), t("like"));
 			$dislike = array( t("I don't like this \x28toggle\x29"), t("dislike"));
 			$reply_to = array( t("Reply on this comment"), t("reply"), t("Reply to"));
 		}
 
+		$share = [];
+		$embed = [];
 		if ($shareable) {
 			// This actually turns out not to be possible in some protocol stacks without opening up hundreds of new issues.
 			// Will allow it only for uri resolvable sources.
 			if(strpos($item['mid'],'http') === 0) {
-				$share = []; //Not yet ready for primetime
+				//Not yet ready for primetime
 				//$share = array( t('Repeat This'), t('repeat'));
 			}
-			$embed = array( t('Share This'), t('share'));
+			$embed = [t('Share This'), t('share')];
 		}
 
 		$dreport = '';
@@ -365,11 +330,13 @@ class ThreadItem {
 		if($keep_reports === 0)
 			$keep_reports = 10;
 
-		if((! get_config('system','disable_dreport')) && strcmp(datetime_convert('UTC','UTC',$item['created']),datetime_convert('UTC','UTC',"now - $keep_reports days")) > 0) {
+		$dreport_link = '';
+		if((intval($item['item_type']) == ITEM_TYPE_POST) && (! get_config('system','disable_dreport')) && strcmp(datetime_convert('UTC','UTC',$item['created']),datetime_convert('UTC','UTC',"now - $keep_reports days")) > 0) {
 			$dreport = t('Delivery Report');
 			$dreport_link = gen_link_id($item['mid']);
 		}
 
+		$is_new = false;
 		if(strcmp(datetime_convert('UTC','UTC',$item['created']),datetime_convert('UTC','UTC','now - 12 hours')) > 0)
 			$is_new = true;
 
@@ -385,8 +352,8 @@ class ThreadItem {
 		if($conv->get_mode() === 'channel')
 			$viewthread = z_root() . '/channel/' . $owner_address . '?f=&mid=' . urlencode(gen_link_id($item['mid']));
 
-		$comment_count_txt = sprintf( tt('%d comment','%d comments',$total_children),$total_children );
-		$list_unseen_txt = (($unseen_comments) ? sprintf( t('%d unseen'),$unseen_comments) : '');
+		$comment_count_txt = sprintf(tt('%d Comment', '%d Comments', $total_children), $total_children);
+		$list_unseen_txt = (($unseen_comments) ? sprintf(t('%d unseen'), $unseen_comments) : '');
 
 		$children = $this->get_children();
 
@@ -439,9 +406,6 @@ class ThreadItem {
 			'author_is_group_actor' => (($item['author']['xchan_pubforum']) ? t('Forum') : ''),
 			'isevent' => $isevent,
 			'attend' => $attend,
-			'consensus' => $consensus,
-			'conlabels' => $conlabels,
-			'canvote' => $canvote,
 			'linktitle' => (($item['author']['xchan_addr']) ? $item['author']['xchan_addr'] : $item['author']['xchan_url']),
 			'olinktitle' => (($item['owner']['xchan_addr']) ? $item['owner']['xchan_addr'] : $item['owner']['xchan_url']),
 			'llink' => $item['llink'],
@@ -510,7 +474,6 @@ class ThreadItem {
 			'bookmark'  => (($conv->get_profile_owner() == local_channel() && local_channel() && $has_bookmarks) ? t('Save Bookmarks') : ''),
 			'addtocal'  => (($has_event) ? t('Add to Calendar') : ''),
 			'drop'      => $drop,
-			'multidrop' => ((feature_enabled($conv->get_profile_owner(),'multi_delete')) ? $multidrop : ''),
 			'dropdown_extras' => $dropdown_extras,
 // end toolbar buttons
 			'unseen_comments' => $unseen_comments,
@@ -533,14 +496,18 @@ class ThreadItem {
 			'modal_dismiss' => t('Close'),
 			'showlike' => $showlike,
 			'showdislike' => $showdislike,
-			'comment' => ($item['item_delayed'] ? '' : $this->get_comment_box($indent)),
+			'comment' => ($item['item_delayed'] ? '' : $this->get_comment_box()),
 			'previewing' => ($conv->is_preview() ? true : false ),
 			'preview_lbl' => t('This is an unsaved preview'),
 			'wait' => t('Please wait'),
 			'thread_level' => $thread_level,
 			'settings' => $settings,
 			'thr_parent' => (($item['parent_mid'] != $item['thr_parent']) ? gen_link_id($item['thr_parent']) : ''),
-			'contact_id' => (($contact) ? $contact['abook_id'] : '')
+			'contact_id' => (($contact) ? $contact['abook_id'] : ''),
+			'moderate' => ($item['item_blocked'] == ITEM_MODERATED),
+			'moderate_approve' => t('Approve'),
+			'moderate_delete' => t('Delete'),
+			'rtl' => in_array($item['lang'], rtl_languages())
 
 		);
 
@@ -827,7 +794,7 @@ class ThreadItem {
 	 *      _ The comment box string (empty if no comment box)
 	 *      _ false on failure
 	 */
-	private function get_comment_box($indent) {
+	private function get_comment_box() {
 
 		if(!$this->is_toplevel() && !get_config('system','thread_allow')) {
 			return '';
@@ -873,7 +840,6 @@ class ThreadItem {
 			'$edurl' => t('Insert Link'),
 			'$edvideo' => t('Video'),
 			'$preview' => t('Preview'), // ((feature_enabled($conv->get_profile_owner(),'preview')) ? t('Preview') : ''),
-			'$indent' => $indent,
 			'$can_upload' => (perm_is_allowed($conv->get_profile_owner(),get_observer_hash(),'write_storage') && $conv->is_uploadable()),
 			'$feature_encrypt' => ((feature_enabled($conv->get_profile_owner(),'content_encrypt')) ? true : false),
 			'$encrypt' => t('Encrypt text'),
@@ -908,7 +874,7 @@ class ThreadItem {
 
 		if($this->is_toplevel() && ($this->get_data_value('author_xchan') != $this->get_data_value('owner_xchan'))) {
 			$this->owner_url = chanlink_hash($this->data['owner']['xchan_hash']);
-			$this->owner_photo = $this->data['owner']['xchan_photo_m'];
+			$this->owner_photo = $this->data['owner']['xchan_photo_s'];
 			$this->owner_name = $this->data['owner']['xchan_name'];
 			$this->wall_to_wall = true;
 		}

Zotlabs/Lib/ThreadListener.php

@@ -4,18 +4,30 @@ namespace Zotlabs\Lib;
 
 class ThreadListener {
 
+	public static function isEnabled() {
+		return Config::Get('system','enable_thread_listener');
+	}
+
 	static public function store($target_id,$portable_id,$ltype = 0) {
+		if (!self::isEnabled()) {
+			return true;
+		}
+
 		$x = self::fetch($target_id,$portable_id,$ltype = 0);
-		if(! $x) {  
+		if(! $x) {
 			$r = q("insert into listeners ( target_id, portable_id, ltype ) values ( '%s', '%s' , %d ) ",
 				dbesc($target_id),
 				dbesc($portable_id),
 				intval($ltype)
 			);
-		}  	
+		}
 	}
 
 	static public function fetch($target_id,$portable_id,$ltype = 0) {
+		if (!self::isEnabled()) {
+			return false;
+		}
+
 		$x = q("select * from listeners where target_id = '%s' and portable_id = '%s' and ltype = %d limit 1",
 			dbesc($target_id),
 			dbesc($portable_id),
@@ -28,6 +40,10 @@ class ThreadListener {
 	}
 
 	static public function fetch_by_target($target_id,$ltype = 0) {
+		if (!self::isEnabled()) {
+			return [];
+		}
+
 		$x = q("select * from listeners where target_id = '%s' and ltype = %d",
 			dbesc($target_id),
 			intval($ltype)

Zotlabs/Lib/ThreadStream.php

@@ -69,12 +69,12 @@ class ThreadStream {
 			case 'cards':
 				$this->profile_owner = \App::$profile['profile_uid'];
 				$this->writable = perm_is_allowed($this->profile_owner,$ob_hash,'post_comments');
-				$this->reload = $_SESSION['return_url'];
+				//$this->reload = $_SESSION['return_url'];
 				break;
 			case 'articles':
 				$this->profile_owner = \App::$profile['profile_uid'];
 				$this->writable = perm_is_allowed($this->profile_owner,$ob_hash,'post_comments');
-				$this->reload = $_SESSION['return_url'];
+				//$this->reload = $_SESSION['return_url'];
 				break;
 			case 'display':
 				// in this mode we set profile_owner after initialisation (from conversation()) and then

Zotlabs/Lib/Webfinger.php

@@ -52,15 +52,21 @@ class Webfinger {
 
 		if(strpos($resource,'http') === 0) {
 			$m = parse_url($resource);
-			if($m) {
-				if($m['scheme'] !== 'https') {
-					return false;
-				}
-				self::$server = $m['host'] . (($m['port']) ? ':' . $m['port'] : '');
-			}
-			else {
+
+			if (!$m) {
 				return false;
 			}
+
+			if(isset($m['scheme']) && $m['scheme'] !== 'https') {
+				return false;
+			}
+
+			if(!isset($m['host'])) {
+				return false;
+			}
+
+			self::$server = $m['host'] . ((isset($m['port'])) ? ':' . $m['port'] : '');
+
 		}
 		elseif(strpos($resource,'tag:') === 0) {
 			$arr = explode(':',$resource); // split the tag
@@ -86,7 +92,7 @@ class Webfinger {
 	/**
 	 * @brief fetch a webfinger resource and return a zot6 discovery url if present
 	 *
-	 */ 
+	 */
 
 	static function zot_url($resource) {
 

Zotlabs/Lib/XConfig.php

@@ -162,7 +162,7 @@ class XConfig {
 	 */
 	static public function Delete($xchan, $family, $key) {
 
-		if(x(\App::$config[$xchan][$family], $key))
+		if(isset(\App::$config[$xchan][$family][$key]))
 			unset(\App::$config[$xchan][$family][$key]);
 
 		$ret = q("DELETE FROM xconfig WHERE xchan = '%s' AND cat = '%s' AND k = '%s'",

Zotlabs/Lib/Zotfinger.php

@@ -35,9 +35,13 @@ class Zotfinger {
 		$result = [];
 
 		$redirects = 0;
-		$x = z_post_url($resource,$data,$redirects, [ 'headers' => $h  ] );
 
-		logger('fetch: ' . print_r($x,true));
+		$start_timestamp = microtime(true);
+		$x = z_post_url($resource,$data,$redirects, [ 'headers' => $h  ] );
+		logger('logger_stats_data cmd:Zotfinger' . ' start:' . $start_timestamp . ' ' . 'end:' . microtime(true) . ' meta:' . $resource . '#' . random_string(16));
+		btlogger('Zotfinger');
+
+		logger('fetch: ' . print_r($x,true), LOGGER_DATA);
 
         if (in_array(intval($x['return_code']), [ 404, 410 ]) && $recurse) {
 
@@ -74,7 +78,7 @@ class Zotfinger {
 				$result['data'] = json_decode(Crypto::unencapsulate($result['data'],get_config('system','prvkey')),true);
 			}
 
-			logger('decrypted: ' . print_r($result,true));
+			logger('decrypted: ' . print_r($result,true), LOGGER_DATA);
 
 			return $result;
 		}

Zotlabs/Module/Acl.php

@@ -66,6 +66,11 @@ class Acl extends \Zotlabs\Web\Controller {
 			killme();
 
 		$permitted = [];
+		$sql_extra = '';
+		$sql_extra2 = '';
+		$sql_extra3 = '';
+		$sql_extra2_xchan = '';
+		$order_extra2 = '';
 
 		if(in_array($type, [ 'm', 'a', 'c', 'f' ])) {
 
@@ -81,7 +86,6 @@ class Acl extends \Zotlabs\Web\Controller {
 
 		}
 
-
 		if($search) {
 			$sql_extra = " AND pgrp.gname LIKE " . protect_sprintf( "'%" . dbesc($search) . "%'" ) . " ";
 			$sql_extra2 = "AND ( xchan_name LIKE " . protect_sprintf( "'%" . dbesc($search) . "%'" ) . " OR xchan_addr LIKE " . protect_sprintf( "'%" . dbesc(punify($search)) . ((strpos($search,'@') === false) ? "%@%'"  : "%'")) . ") ";
@@ -100,10 +104,6 @@ class Acl extends \Zotlabs\Web\Controller {
 			$sql_extra3 = "AND ( xchan_addr like " . protect_sprintf( "'%" . dbesc(punify($search)) . "%'" ) . " OR xchan_name like " . protect_sprintf( "'%" . dbesc($search) . "%'" ) . " ) ";
 
 		}
-		else {
-			$sql_extra = $sql_extra2 = $sql_extra3 = "";
-		}
-
 
 		$groups = array();
 		$contacts = array();
@@ -284,7 +284,7 @@ class Acl extends \Zotlabs\Web\Controller {
 		}
 		elseif($type == 'm') {
 			$r = array();
-			$z = q("SELECT xchan_hash as hash, xchan_name as name, xchan_network as net, xchan_addr as nick, xchan_photo_s as micro, xchan_url as url
+			$z = q("SELECT abook_id as id, xchan_hash as hash, xchan_name as name, xchan_network as net, xchan_addr as nick, xchan_photo_s as micro, xchan_url as url, abook_self
 				FROM abook left join xchan on abook_xchan = xchan_hash
 				WHERE abook_channel = %d
 				and xchan_deleted = 0
@@ -304,7 +304,7 @@ class Acl extends \Zotlabs\Web\Controller {
 		}
 		elseif($type == 'a') {
 
-			$r = q("SELECT abook_id as id, xchan_name as name, xchan_network as net, xchan_hash as hash, xchan_addr as nick, xchan_photo_s as micro, xchan_url as url, xchan_addr as attag , abook_their_perms FROM abook left join xchan on abook_xchan = xchan_hash
+			$r = q("SELECT abook_id as id, xchan_name as name, xchan_network as net, xchan_hash as hash, xchan_addr as nick, xchan_photo_s as micro, xchan_url as url, xchan_addr as attag, abook_their_perms, abook_self FROM abook left join xchan on abook_xchan = xchan_hash
 				WHERE abook_channel = %d
 				and xchan_deleted = 0
 				$sql_extra3
@@ -342,7 +342,7 @@ class Acl extends \Zotlabs\Web\Controller {
 			$x = [];
 			foreach($r as $g) {
 
-				if(in_array($g['net'],['rss','anon','unknown']) && ($type != 'a'))
+				if(isset($g['net']) && in_array($g['net'], ['rss','anon','unknown']) && ($type != 'a'))
 					continue;
 
 				$g['hash'] = urlencode($g['hash']);
@@ -378,12 +378,13 @@ class Acl extends \Zotlabs\Web\Controller {
 							"name"     => $g['name'],
 							"id"	   => urlencode($g['id']),
 							"xid"      => $g['hash'],
+							"url"      => $g['url'],
 							"link"     => $clink,
 							"nick"     => ((strpos($g['nick'],'@')) ? substr($g['nick'],0,strpos($g['nick'],'@')) : $g['nick']),
 							"self"     => (intval($g['abook_self']) ? 'abook-self' : ''),
 							"taggable" => '',
 							"label"    => '',
-							"net"      => $g['net']
+							"net"      => $g['net'] ?? ''
 						);
 					}
 				}

Zotlabs/Module/Activity.php

@@ -182,7 +182,7 @@ class Activity extends Controller {
 				return;
 			}
 
-			$ob_authorise = false;
+			$ob_authorize = false;
 			$item_uid = 0;
 
 			$bear = ZlibActivity::token_from_request();

Zotlabs/Module/Admin/Accounts.php

@@ -346,7 +346,7 @@ class Accounts {
 			'$users' 	=> $users,
 			'$msg'      => t('Message')
 		));
-		$o .= paginate($a);
+		$o .= paginate();
 
 		return $o;
 	}

Zotlabs/Module/Admin/Addons.php

@@ -77,6 +77,7 @@ class Addons {
 					} catch (\PHPGit\Exception\GitException $e) {
 						json_return_and_die(array('message' => 'Error updating addon repo.', 'success' => false));
 					}
+					break;
 				case 'removerepo':
 					if (array_key_exists('repoName', $_REQUEST)) {
 						$repoName = $_REQUEST['repoName'];
@@ -111,6 +112,7 @@ class Addons {
 					} else {
 						json_return_and_die(array('message' => 'Error deleting addon repo.', 'success' => false));
 					}
+					break;
 				case 'installrepo':
 					if (array_key_exists('repoURL', $_REQUEST)) {
 						require_once('library/PHPGit.autoload.php');			// Load PHPGit dependencies
@@ -172,6 +174,7 @@ class Addons {
 						$repo = $git->probeRepo();
 						json_return_and_die(array('repo' => $repo, 'message' => '', 'success' => true));
 					}
+					break;
 				case 'addrepo':
 					if (array_key_exists('repoURL', $_REQUEST)) {
 						require_once('library/PHPGit.autoload.php');	 // Load PHPGit dependencies

Zotlabs/Module/Admin/Channels.php

@@ -168,7 +168,7 @@ class Channels {
 			'$baseurl' => z_root(),
 			'$channels' => $channels,
 		));
-		$o .= paginate($a);
+		$o .= paginate();
 
 		return $o;
 	}

Zotlabs/Module/Admin/Dbsync.php

@@ -5,11 +5,11 @@ namespace Zotlabs\Module\Admin;
 
 
 class Dbsync {
-	
+
 
 	function get() {
 		$o = '';
-	
+
 		if(argc() > 3 && intval(argv(3)) && argv(2) === 'mark') {
 			// remove the old style config if it exists
 			del_config('database', 'update_r' . intval(argv(3)));
@@ -29,7 +29,7 @@ class Dbsync {
 				if(method_exists($c,'verify')) {
 					$retval = $c->verify();
 					if($retval === UPDATE_FAILED) {
-						$o .= sprintf( t('Verification of update %s failed. Check system logs.'), $s); 
+						$o .= sprintf( t('Verification of update %s failed. Check system logs.'), $s);
 					}
 					elseif($retval === UPDATE_SUCCESS) {
 						$o .= sprintf( t('Update %s was successfully applied.'), $s);
@@ -44,20 +44,8 @@ class Dbsync {
 			}
 			else
 				$o .= sprintf( t('Update function %s could not be found.'), $s);
-	
+
 			return $o;
-
-
-
-
-
-			// remove the old style config if it exists
-			del_config('database', 'update_r' . intval(argv(3)));
-			set_config('database', '_' . intval(argv(3)), 'success');
-			if(intval(get_config('system','db_version')) < intval(argv(3)))
-				set_config('system','db_version',intval(argv(3)));
-			info( t('Update has been marked successful') . EOL);
-			goaway(z_root() . '/admin/dbsync');
 		}
 
 		if(argc() > 2 && intval(argv(2))) {
@@ -68,7 +56,7 @@ class Dbsync {
 				$c =  new $cls();
 				$retval = $c->run();
 				if($retval === UPDATE_FAILED) {
-					$o .= sprintf( t('Executing update procedure %s failed. Check system logs.'), $s); 
+					$o .= sprintf( t('Executing update procedure %s failed. Check system logs.'), $s);
 				}
 				elseif($retval === UPDATE_SUCCESS) {
 					$o .= sprintf( t('Update %s was successfully applied.'), $s);
@@ -79,10 +67,10 @@ class Dbsync {
 			}
 			else
 				$o .= sprintf( t('Update function %s could not be found.'), $s);
-	
+
 			return $o;
 		}
-	
+
 		$failed = array();
 		$r = q("select * from config where cat = 'database' ");
 		if(count($r)) {
@@ -107,7 +95,7 @@ class Dbsync {
 		else {
 			return '<div class="generic-content-wrapper-styled"><h3>' . t('No failed updates.') . '</h3></div>';
 		}
-	
+
 		return $o;
 	}
-}
+}

Zotlabs/Module/Admin/Queue.php

@@ -12,18 +12,18 @@ class Queue {
 
 		$o = '';
 
-		$expert = ((array_key_exists('expert',$_REQUEST)) ? intval($_REQUEST['expert']) : 0);
+		$expert = $_REQUEST['expert'] ?? false;
 
-		if($_REQUEST['drophub']) {
+		if(isset($_REQUEST['drophub'])) {
 			hubloc_mark_as_down($_REQUEST['drophub']);
 			LibQueue::remove_by_posturl($_REQUEST['drophub']);
 		}
 
-		if($_REQUEST['emptyhub']) {
+		if(isset($_REQUEST['emptyhub'])) {
 			LibQueue::remove_by_posturl($_REQUEST['emptyhub']);
 		}
 
-		if($_REQUEST['deliverhub']) {
+		if(isset($_REQUEST['deliverhub'])) {
 
 			$hubq = q("SELECT * FROM outq WHERE outq_posturl = '%s'",
 				dbesc($_REQUEST['deliverhub'])
@@ -39,7 +39,6 @@ class Queue {
 
 		for($x = 0; $x < count($r); $x ++) {
 			$r[$x]['eurl'] = urlencode($r[$x]['outq_posturl']);
-			$r[$x]['connected'] = datetime_convert('UTC',date_default_timezone_get(),$r[$x]['connected'],'Y-m-d');
 		}
 
 		$o = replace_macros(get_markup_template('admin_queue.tpl'), array(

Zotlabs/Module/Admin/Queueworker.php

@@ -0,0 +1,121 @@
+<?php
+
+namespace Zotlabs\Module\Admin;
+
+use App;
+use Zotlabs\Web\Controller;
+
+class Queueworker extends Controller {
+
+	function init() {
+
+	}
+
+	function post() {
+
+		check_form_security_token('form_security_token', 'queueworker');
+
+		$maxqueueworkers = intval($_POST['queueworker_maxworkers']);
+		$maxqueueworkers = ($maxqueueworkers > 3) ? $maxqueueworkers : 4;
+		set_config('queueworker', 'max_queueworkers', $maxqueueworkers);
+
+		$maxworkerage = intval($_POST['queueworker_max_age']);
+		$maxworkerage = ($maxworkerage >= 120) ? $maxworkerage : 300;
+		set_config('queueworker', 'queueworker_max_age', $maxworkerage);
+
+		$queueworkersleep = intval($_POST['queue_worker_sleep']);
+		$queueworkersleep = ($queueworkersleep > 100) ? $queueworkersleep : 100;
+		set_config('queueworker', 'queue_worker_sleep', $queueworkersleep);
+
+		$auto_queue_worker_sleep = intval($_POST['auto_queue_worker_sleep']);
+		set_config('queueworker', 'auto_queue_worker_sleep', $auto_queue_worker_sleep);
+
+		goaway(z_root() . '/admin/queueworker');
+	}
+
+	function get() {
+
+		$content = "<H1>Queue Status</H1>\n";
+
+		$r = q('select count(*) as total from workerq');
+
+		$content .= "<H4>There are " . $r[0]['total'] . " queue items to be processed.</H4>";
+
+		$r = dbq("select count(*) as qworkers from workerq where workerq_reservationid is not null");
+
+		$content .= "<H4>Active workers: " . $r[0]['qworkers'] . "</H4>";
+
+		$r = dbq("select workerq_cmd, count(*) as total from workerq where true group by workerq_cmd");
+
+		if ($r) {
+			$content .= "<H4>Work items</H4>";
+			foreach($r as $rr) {
+				$content .= $rr['workerq_cmd'] . ': ' . $rr['total'] . '<br>';
+			}
+		}
+
+		$maxqueueworkers = get_config('queueworker', 'max_queueworkers', 4);
+		$maxqueueworkers = ($maxqueueworkers > 3) ? $maxqueueworkers : 4;
+
+		$sc = '';
+
+		$sc .= replace_macros(get_markup_template('field_input.tpl'), [
+			'$field' => [
+				'queueworker_maxworkers',
+				t('Max queueworker threads'),
+				$maxqueueworkers,
+				t('Minimum 4, default 4')
+			]
+		]);
+
+		$workermaxage = get_config('queueworker', 'queueworker_max_age');
+		$workermaxage = ($workermaxage >= 120) ? $workermaxage : 300;
+
+		$sc .= replace_macros(get_markup_template('field_input.tpl'), [
+			'$field' => [
+				'queueworker_max_age',
+				t('Assume workers dead after'),
+				$workermaxage,
+				t('Minimum 120, default 300 seconds')
+			]
+		]);
+
+		$queueworkersleep = get_config('queueworker', 'queue_worker_sleep');
+		$queueworkersleep = ($queueworkersleep > 100) ? $queueworkersleep : 100;
+
+		$auto_queue_worker_sleep = get_config('queueworker', 'auto_queue_worker_sleep', 0);
+
+		$sc .= replace_macros(get_markup_template('field_input.tpl'), [
+			'$field' => [
+				'queue_worker_sleep',
+				t('Pause before starting next task'),
+				$queueworkersleep,
+				t('Minimum 100, default 100 microseconds'),
+				'',
+				(($auto_queue_worker_sleep) ? 'disabled' : '')
+			]
+		]);
+
+		$sc .= replace_macros(get_markup_template('field_checkbox.tpl'), [
+			'$field' => [
+				'auto_queue_worker_sleep',
+				t('Automatically adjust pause before starting next task'),
+				$auto_queue_worker_sleep,
+			]
+		]);
+
+		$tpl = get_markup_template('settings_addon.tpl');
+		$content .= replace_macros($tpl, [
+				'$action_url' => 'admin/queueworker',
+				'$form_security_token' => get_form_security_token('queueworker'),
+				'$title' => t('Queueworker Settings'),
+				'$content' => $sc,
+				'$baseurl' => z_root(),
+				'$submit' => t('Save')
+			]
+		);
+
+		return $content;
+
+	}
+}

Zotlabs/Module/Admin/Security.php

@@ -7,12 +7,12 @@ class Security {
 
 	function post() {
 		check_form_security_token_redirectOnErr('/admin/security', 'admin_security');
-	
+
 		$allowed_email        = ((x($_POST,'allowed_email'))	    ? notags(trim($_POST['allowed_email']))		: '');
 		$not_allowed_email    = ((x($_POST,'not_allowed_email'))	? notags(trim($_POST['not_allowed_email']))		: '');
 
 		set_config('system','allowed_email', $allowed_email);
-		set_config('system','not_allowed_email', $not_allowed_email);	
+		set_config('system','not_allowed_email', $not_allowed_email);
 
 		$block_public         = ((x($_POST,'block_public'))		? True	: False);
 		set_config('system','block_public',$block_public);
@@ -25,22 +25,22 @@ class Security {
 
 		$ws = $this->trim_array_elems(explode("\n",$_POST['whitelisted_sites']));
 		set_config('system','whitelisted_sites',$ws);
-	
+
 		$bs = $this->trim_array_elems(explode("\n",$_POST['blacklisted_sites']));
 		set_config('system','blacklisted_sites',$bs);
-	
+
 		$wc = $this->trim_array_elems(explode("\n",$_POST['whitelisted_channels']));
 		set_config('system','whitelisted_channels',$wc);
-	
+
 		$bc = $this->trim_array_elems(explode("\n",$_POST['blacklisted_channels']));
 		set_config('system','blacklisted_channels',$bc);
-	
+
 		$embed_sslonly         = ((x($_POST,'embed_sslonly'))		? True	: False);
 		set_config('system','embed_sslonly',$embed_sslonly);
-	
+
 		$we = $this->trim_array_elems(explode("\n",$_POST['embed_allow']));
 		set_config('system','embed_allow',$we);
-	
+
 		$be = $this->trim_array_elems(explode("\n",$_POST['embed_deny']));
 		set_config('system','embed_deny',$be);
 
@@ -49,47 +49,54 @@ class Security {
 
 		$inline_pdf = ((x($_POST,'inline_pdf')) ? intval($_POST['inline_pdf']) : 0);
 		set_config('system', 'inline_pdf' , $inline_pdf);
-	
+
 		$ts = ((x($_POST,'transport_security')) ? True : False);
 		set_config('system','transport_security_header',$ts);
 
 		$cs = ((x($_POST,'content_security')) ? True : False);
 		set_config('system','content_security_policy',$cs);
 
+		$trusted_directory_servers = $this->trim_array_elems(explode("\n", $_POST['trusted_directory_servers']));
+		set_config('system', 'trusted_directory_servers', $trusted_directory_servers);
+
 		goaway(z_root() . '/admin/security');
 	}
-	
-	
+
+
 
 	function get() {
-	
+
 		$whitesites = get_config('system','whitelisted_sites');
 		$whitesites_str = ((is_array($whitesites)) ? implode("\n",$whitesites) : '');
-	
+
 		$blacksites = get_config('system','blacklisted_sites');
 		$blacksites_str = ((is_array($blacksites)) ? implode("\n",$blacksites) : '');
-	
-	
+
+
 		$whitechannels = get_config('system','whitelisted_channels');
 		$whitechannels_str = ((is_array($whitechannels)) ? implode("\n",$whitechannels) : '');
-	
+
 		$blackchannels = get_config('system','blacklisted_channels');
 		$blackchannels_str = ((is_array($blackchannels)) ? implode("\n",$blackchannels) : '');
-	
-	
+
+
 		$whiteembeds = get_config('system','embed_allow');
 		$whiteembeds_str = ((is_array($whiteembeds)) ? implode("\n",$whiteembeds) : '');
-	
+
 		$blackembeds = get_config('system','embed_deny');
 		$blackembeds_str = ((is_array($blackembeds)) ? implode("\n",$blackembeds) : '');
-	
+
+		$trusted_directory_servers = get_config('system', 'trusted_directory_servers');
+		$trusted_directory_servers_str = ((is_array($trusted_directory_servers)) ? implode("\n", $trusted_directory_servers) : '');
+		$is_dir = (intval(get_config('system', 'directory_mode', DIRECTORY_MODE_NORMAL)) !== DIRECTORY_MODE_NORMAL);
+
 		$embed_coop = intval(get_config('system','embed_coop'));
-	
+
 		if((! $whiteembeds) && (! $blackembeds)) {
 			$embedhelp1 = t("By default, unfiltered HTML is allowed in embedded media. This is inherently insecure.");
 		}
 
-		$embedhelp2 = t("The recommended setting is to only allow unfiltered HTML from the following sites:"); 
+		$embedhelp2 = t("The recommended setting is to only allow unfiltered HTML from the following sites:");
 		$embedhelp3 = t("https://youtube.com/<br />https://www.youtube.com/<br />https://youtu.be/<br />https://vimeo.com/<br />https://soundcloud.com/<br />");
 		$embedhelp4 = t("All other embedded content will be filtered, <strong>unless</strong> embedded content from that site is explicitly blocked.");
 
@@ -99,7 +106,7 @@ class Security {
 			'$page' => t('Security'),
 			'$form_security_token' => get_form_security_token('admin_security'),
 	        '$block_public'     => array('block_public', t("Block public"), get_config('system','block_public'), t("Check to block public access to all otherwise public personal pages on this site unless you are currently authenticated.")),
-			'$cloud_noroot'     => [ 'cloud_noroot', t('Provide a cloud root directory'), 1 - intval(get_config('system','cloud_disable_siteroot')), t('The cloud root directory lists all channel names which provide public files') ], 
+			'$cloud_noroot'     => [ 'cloud_noroot', t('Provide a cloud root directory'), 1 - intval(get_config('system','cloud_disable_siteroot')), t('The cloud root directory lists all channel names which provide public files') ],
 			'$cloud_disksize'     => [ 'cloud_disksize', t('Show total disk space available to cloud uploads'), intval(get_config('system','cloud_report_disksize')), '' ],
 			'$transport_security' => array('transport_security', t('Set "Transport Security" HTTP header'),intval(get_config('system','transport_security_header')),''),
 			'$content_security' => array('content_security', t('Set "Content Security Policy" HTTP header'),intval(get_config('system','content_security_policy')),''),
@@ -115,6 +122,8 @@ class Security {
 			'$thumbnail_security'   => [ 'thumbnail_security', t("Allow SVG thumbnails in file browser"), get_config('system','thumbnail_security',0), t("WARNING: SVG images may contain malicious code.") ],
 			'$inline_pdf'   => [ 'inline_pdf', t("Allow embedded (inline) PDF files"), get_config('system','inline_pdf',0), '' ],
 
+			'$trusted_directory_servers' => (($is_dir) ? ['trusted_directory_servers', t('Additional trusted directory server URLs'), $trusted_directory_servers_str, t('Accept directory flags (spam, nsfw) from those servers. One per line like https://example.tld')] : ''),
+
 //	        '$embed_coop'     => array('embed_coop', t('Cooperative embed security'), $embed_coop, t('Enable to share embed security with other compatible sites/hubs')),
 
 			'$submit' => t('Submit')
@@ -124,7 +133,7 @@ class Security {
 
 	function trim_array_elems($arr) {
 		$narr = array();
-	
+
 		if($arr && is_array($arr)) {
 			for($x = 0; $x < count($arr); $x ++) {
 				$y = trim($arr[$x]);
@@ -134,6 +143,6 @@ class Security {
 		}
 		return $narr;
 	}
-	
-	
+
+
 }

Zotlabs/Module/Admin/Site.php

@@ -67,7 +67,6 @@ class Site {
 		$open_pubstream     =   ((x($_POST,'open_pubstream')) ? True : False);
 		$login_on_homepage	=	((x($_POST,'login_on_homepage'))		? True	:	False);
 		$enable_context_help = ((x($_POST,'enable_context_help'))		? True	:	False);
-		$global_directory     = ((x($_POST,'directory_submit_url'))	? notags(trim($_POST['directory_submit_url']))	: '');
 		$no_community_page    = !((x($_POST,'no_community_page'))	? True	:	False);
 		$default_expire_days  = ((array_key_exists('default_expire_days',$_POST)) ? intval($_POST['default_expire_days']) : 0);
 		$active_expire_days  = ((array_key_exists('active_expire_days',$_POST)) ? intval($_POST['active_expire_days']) : 7);
@@ -100,7 +99,7 @@ class Site {
 		$reg_expire 	   = (preg_match('/^[a-z]{1,1}$/', $regexpireu) ? $regexpiren . $regexpireu : '');
 
 		$imagick_path      = ((x($_POST,'imagick_path'))     ? trim($_POST['imagick_path'])   : '');
-		$force_queue       = ((intval($_POST['force_queue']) > 0) ? intval($_POST['force_queue'])   : 3000);
+		//$force_queue       = ((intval($_POST['force_queue']) > 0) ? intval($_POST['force_queue'])   : 3000);
 		$pub_incl = escape_tags(trim($_POST['pub_incl']));
 		$pub_excl = escape_tags(trim($_POST['pub_excl']));
 
@@ -129,7 +128,7 @@ class Site {
 					set_config('system', 'register_duty', $this->register_duty);
 					set_config('system', 'register_duty_jso', $this->joo);
 				} else {
-					notice('ZAR0130E,'.t('Errors') . ': ' . $this->error) . EOL . $this->msgfg;
+					notice('ZAR0130E,' . t('Errors') . ': ' . $this->error . EOL . $this->msgfg . EOL);
 				}
 			}
 		}
@@ -181,7 +180,7 @@ class Site {
 			set_config('system', 'admininfo', $admininfo);
 		}
 		set_config('system','siteinfo',$siteinfo);
-		set_config('system', 'language', $language);
+		//set_config('system', 'language', $language);
 		set_config('system', 'theme', $theme);
 		//		if ( $theme_mobile === '---' ) {
 		//			del_config('system', 'mobile_theme');
@@ -205,12 +204,7 @@ class Site {
 		set_config('system','disable_discover_tab', $disable_discover_tab);
 		set_config('system','site_firehose', $site_firehose);
 		set_config('system','open_pubstream', $open_pubstream);
-		set_config('system','force_queue_threshold', $force_queue);
-		if ($global_directory == '') {
-			del_config('system', 'directory_submit_url');
-		} else {
-			set_config('system', 'directory_submit_url', $global_directory);
-		}
+		//set_config('system','force_queue_threshold', $force_queue);
 
 		set_config('system','no_community_page', $no_community_page);
 		set_config('system','no_utf', $no_utf);
@@ -283,7 +277,7 @@ class Site {
 		}
 
 		$dir_choices = null;
-		$dirmode = get_config('system','directory_mode');
+		$dirmode = get_config('system', 'directory_mode', DIRECTORY_MODE_NORMAL);
 		$realm = get_directory_realm();
 
 		// directory server should not be set or settable unless we are a directory client
@@ -301,6 +295,12 @@ class Site {
 					$dir_choices[$xx['site_url']] = $xx['site_url'];
 				}
 			}
+			if ($realm === DIRECTORY_REALM) {
+				$fallback_servers = get_directory_fallback_servers();
+				foreach ($fallback_servers as $fallback_server) {
+					$dir_choices[$fallback_server] = $fallback_server;
+				}
+			}
 		}
 
 		/* Banner */
@@ -425,7 +425,7 @@ class Site {
 			'$banner'			=> array('banner', t("Banner/Logo"), $banner, t('Unfiltered HTML/CSS/JS is allowed')),
 			'$admininfo'		=> array('admininfo', t("Administrator Information"), $admininfo, t("Contact information for site administrators.  Displayed on siteinfo page.  BBCode can be used here")),
 			'$siteinfo'		=> array('siteinfo', t('Site Information'), get_config('system','siteinfo'), t("Publicly visible description of this site.  Displayed on siteinfo page.  BBCode can be used here")),
-			'$language' 		=> array('language', t("System language"), get_config('system','language'), "", $lang_choices),
+			//'$language' 		=> array('language', t("System language"), get_config('system','language'), "", $lang_choices),
 			'$theme' 			=> array('theme', t("System theme"), get_config('system','theme'), t("Default system theme - may be over-ridden by user profiles - <a href='#' id='cnftheme'>change theme settings</a>"), $theme_choices),
 		//			'$theme_mobile' 	=> array('theme_mobile', t("Mobile system theme"), get_config('system','mobile_theme'), t("Theme for mobile devices"), $theme_choices_mobile),
 		//			'$site_channel' 	=> array('site_channel', t("Channel to use for this website's static pages"), get_config('system','site_channel'), t("Site Channel")),
@@ -508,10 +508,10 @@ class Site {
 			'$frontpage'	=> array('frontpage', t("Site homepage to show visitors (default: login box)"), get_config('system','frontpage'), t("example: 'pubstream' to show public stream, 'page/sys/home' to show a system webpage called 'home' or 'include:home.html' to include a file.")),
 			'$mirror_frontpage'	=> array('mirror_frontpage', t("Preserve site homepage URL"), get_config('system','mirror_frontpage'), t('Present the site homepage in a frame at the original location instead of redirecting')),
 			'$allowed_sites'	=> array('allowed_sites', t("Allowed friend domains"), get_config('system','allowed_sites'), t("Comma separated list of domains which are allowed to establish friendships with this site. Wildcards are accepted. Empty to allow any domains")),
-			'$force_publish'	=> array('publish_all', t("Force publish"), get_config('system','publish_all'), t("Check to force all profiles on this site to be listed in the site directory.")),
-			'$disable_discover_tab'	=> array('disable_discover_tab', t('Import Public Streams'), $discover_tab, t('Import and allow access to public content pulled from other sites. Warning: this content is unmoderated.')),
-			'$site_firehose'	=> array('site_firehose', t('Site only Public Streams'), get_config('system','site_firehose'), t('Allow access to public content originating only from this site if Imported Public Streams are disabled.')),
-			'$open_pubstream'	=> array('open_pubstream', t('Allow anybody on the internet to access the Public streams'), get_config('system','open_pubstream',1), t('Disable to require authentication before viewing. Warning: this content is unmoderated.')),
+			'$force_publish'	=> array('publish_all', t("Force publish"), get_config('system','publish_all'), t("Check to force all profiles on this site to be listed in the site directory")),
+			'$disable_discover_tab'	=> array('disable_discover_tab', t('Enable public stream'), $discover_tab, t('Enable the public stream. Warning: this content is unmoderated')),
+			'$site_firehose'	=> array('site_firehose', t('Site only public stream'), get_config('system','site_firehose'), t('Restrict the public stream to content originating at this site')),
+			'$open_pubstream'	=> array('open_pubstream', t('Allow anybody on the internet to access the public streams'), get_config('system','open_pubstream',1), t('Disable to require authentication before viewing')),
 			'$incl'           => array('pub_incl',t('Only import Public stream posts with this text'), get_config('system','pubstream_incl'),t('words one per line or #tags or /patterns/ or lang=xx, leave blank to import all posts')),
 			'$excl'           => array('pub_excl',t('Do not import Public stream posts with this text'), get_config('system','pubstream_excl'),t('words one per line or #tags or /patterns/ or lang=xx, leave blank to import all posts')),
 
@@ -532,7 +532,7 @@ class Site {
 			'$timeout'			=> array('timeout', t("Network timeout"), (x(get_config('system','curl_timeout'))?get_config('system','curl_timeout'):60), t("Value is in seconds. Set to 0 for unlimited (not recommended).")),
 			'$delivery_interval'			=> array('delivery_interval', t("Delivery interval"), (x(get_config('system','delivery_interval'))?get_config('system','delivery_interval'):2), t("Delay background delivery processes by this many seconds to reduce system load. Recommend: 4-5 for shared hosts, 2-3 for virtual private servers. 0-1 for large dedicated servers.")),
 			'$delivery_batch_count' => array('delivery_batch_count', t('Deliveries per process'),(x(get_config('system','delivery_batch_count'))?get_config('system','delivery_batch_count'):1), t("Number of deliveries to attempt in a single operating system process. Adjust if necessary to tune system performance. Recommend: 1-5.")),
-			'$force_queue'			=> array('force_queue', t("Queue Threshold"), get_config('system','force_queue_threshold',3000), t("Always defer immediate delivery if queue contains more than this number of entries.")),
+			//'$force_queue'			=> array('force_queue', t("Queue Threshold"), get_config('system','force_queue_threshold',3000), t("Always defer immediate delivery if queue contains more than this number of entries.")),
 			'$poll_interval'			=> array('poll_interval', t("Poll interval"), (x(get_config('system','poll_interval'))?get_config('system','poll_interval'):2), t("Delay background polling processes by this many seconds to reduce system load. If 0, use delivery interval.")),
 			'$imagick_path'			=> array('imagick_path', t("Path to ImageMagick convert program"), get_config('system','imagick_convert_path'), t("If set, use this program to generate photo thumbnails for huge images ( > 4000 pixels in either dimension), otherwise memory exhaustion may occur. Example: /usr/bin/convert")),
 			'$maxloadavg'			=> array('maxloadavg', t("Maximum Load Average"), ((intval(get_config('system','maxloadavg')) > 0)?get_config('system','maxloadavg'):50), t("Maximum system load before delivery and poll processes are deferred - default 50.")),

Zotlabs/Module/Admin/Themes.php

@@ -80,10 +80,25 @@ class Themes {
 
 				$this->toggle_theme($themes, $theme, $result);
 				$s = $this->rebuild_theme_table($themes);
-				if($result)
-					info( sprintf('Theme %s enabled.', $theme));
-				else
-					info( sprintf('Theme %s disabled.', $theme));
+
+				if($result) {
+					if (is_file("view/theme/$theme/php/config.php")){
+						require_once("view/theme/$theme/php/config.php");
+						if (function_exists($theme . '_theme_admin_enable')){
+							call_user_func($theme . '_theme_admin_enable');
+						}
+					}
+					info(sprintf('Theme %s enabled.', $theme));
+				}
+				else {
+					if (is_file("view/theme/$theme/php/config.php")){
+						require_once("view/theme/$theme/php/config.php");
+						if (function_exists($theme . '_theme_admin_disable')){
+							call_user_func($theme . '_theme_admin_disable');
+						}
+					}
+					info(sprintf('Theme %s disabled.', $theme));
+				}
 
 				set_config('system', 'allowed_themes', $s);
 				goaway(z_root() . '/admin/themes' );

Zotlabs/Module/Appman.php

@@ -13,7 +13,7 @@ class Appman extends \Zotlabs\Web\Controller {
 		if(! local_channel())
 			return;
 
-		if($_POST['url']) {
+		if(isset($_POST['url']) && $_POST['url']) {
 			$arr = array(
 				'uid' => intval($_REQUEST['uid']),
 				'url' => escape_tags($_REQUEST['url']),
@@ -50,7 +50,7 @@ class Appman extends \Zotlabs\Web\Controller {
 			return;
 		}
 
-		if($_POST['install']) {
+		if(isset($_POST['install']) && $_POST['install']) {
 			Apps::app_install(local_channel(),$papp);
 			if(Apps::app_installed(local_channel(),$papp))
 				info( t('App installed.') . EOL);
@@ -65,15 +65,15 @@ class Appman extends \Zotlabs\Web\Controller {
 			}
 
 			if (intval($sync[0]['app_system'])) {
-				Libsync::build_sync_packet($uid, ['sysapp' => $sync]);
+				Libsync::build_sync_packet(local_channel(), ['sysapp' => $sync]);
 			}
 			else {
-				Libsync::build_sync_packet($uid, ['app' => $sync]);
+				Libsync::build_sync_packet(local_channel(), ['app' => $sync]);
 			}
 
 		}
 
-		if($_POST['delete']) {
+		if(isset($_POST['delete']) && $_POST['delete']) {
 
 			// Fetch the app for sync before it is deleted (if it is deletable))
 			$sync = q("SELECT * FROM app WHERE app_channel = %d AND app_id = '%s' LIMIT 1",
@@ -91,18 +91,18 @@ class Appman extends \Zotlabs\Web\Controller {
 			$sync[0]['app_deleted'] = 1;
 
 			if (intval($sync[0]['app_system'])) {
-				Libsync::build_sync_packet($uid, ['sysapp' => $sync]);
+				Libsync::build_sync_packet(local_channel(), ['sysapp' => $sync]);
 			}
 			else {
-				Libsync::build_sync_packet($uid, ['app' => $sync]);
+				Libsync::build_sync_packet(local_channel(), ['app' => $sync]);
 			}
 		}
 
-		if($_POST['edit']) {
+		if(isset($_POST['edit']) && $_POST['edit']) {
 			return;
 		}
 
-		if($_POST['feature']) {
+		if(isset($_POST['feature']) && $_POST['feature']) {
 			Apps::app_feature(local_channel(), $papp, $_POST['feature']);
 
 			$sync = q("SELECT * FROM app WHERE app_channel = %d AND app_id = '%s' LIMIT 1",
@@ -111,14 +111,14 @@ class Appman extends \Zotlabs\Web\Controller {
 			);
 
 			if (intval($sync[0]['app_system'])) {
-				Libsync::build_sync_packet($uid, ['sysapp' => $sync]);
+				Libsync::build_sync_packet(local_channel(), ['sysapp' => $sync]);
 			}
 			else {
-				Libsync::build_sync_packet($uid, ['app' => $sync]);
+				Libsync::build_sync_packet(local_channel(), ['app' => $sync]);
 			}
 		}
 
-		if($_POST['pin']) {
+		if(isset($_POST['pin']) && $_POST['pin']) {
 			Apps::app_feature(local_channel(), $papp, $_POST['pin']);
 
 			$sync = q("SELECT * FROM app WHERE app_channel = %d AND app_id = '%s' LIMIT 1",
@@ -127,14 +127,14 @@ class Appman extends \Zotlabs\Web\Controller {
 			);
 
 			if (intval($sync[0]['app_system'])) {
-				Libsync::build_sync_packet($uid, ['sysapp' => $sync]);
+				Libsync::build_sync_packet(local_channel(), ['sysapp' => $sync]);
 			}
 			else {
-				Libsync::build_sync_packet($uid, ['app' => $sync]);
+				Libsync::build_sync_packet(local_channel(), ['app' => $sync]);
 			}
 		}
 
-		if($_POST['aj']) {
+		if(isset($_POST['aj']) && $_POST['aj']) {
 			killme();
 		}
 
@@ -171,7 +171,7 @@ class Appman extends \Zotlabs\Web\Controller {
 
 		$app = null;
 		$embed = null;
-		if($_REQUEST['appid']) {
+		if(isset($_REQUEST['appid']) && $_REQUEST['appid']) {
 			$r = q("select * from app where app_id = '%s' and app_channel = %d limit 1",
 				dbesc($_REQUEST['appid']),
 				dbesc(local_channel())
@@ -200,27 +200,25 @@ class Appman extends \Zotlabs\Web\Controller {
 		}
 
 		return replace_macros(get_markup_template('app_create.tpl'), array(
-
 			'$banner' => (($app) ? t('Edit App') : t('Create App')),
 			'$app' => $app,
-			'$guid' => (($app) ? $app['app_id'] : ''),
-			'$author' => (($app) ? $app['app_author'] : $channel['channel_hash']),
-			'$addr' => (($app) ? $app['app_addr'] : $channel['xchan_addr']),
-			'$name' => array('name', t('Name of app'),(($app) ? $app['app_name'] : ''), t('Required')),
-			'$url' => array('url', t('Location (URL) of app'),(($app) ? $app['app_url'] : ''), t('Required')),
-	 		'$desc' => array('desc', t('Description'),(($app) ? $app['app_desc'] : ''), ''),
-			'$photo' => array('photo', t('Photo icon URL'),(($app) ? $app['app_photo'] : ''), t('80 x 80 pixels - optional')),
-			'$categories' => array('categories',t('Categories (optional, comma separated list)'),(($app) ? $app['categories'] : ''),''),
-			'$version' => array('version', t('Version ID'),(($app) ? $app['app_version'] : ''), ''),
-			'$price' => array('price', t('Price of app'),(($app) ? $app['app_price'] : ''), ''),
-			'$page' => array('page', t('Location (URL) to purchase app'),(($app) ? $app['app_page'] : ''), ''),
-			'$system' => (($app) ? intval($app['app_system']) : 0),
-			'$plugin' => (($app) ? $app['app_plugin'] : ''),
-			'$requires' => (($app) ? $app['app_requires'] : ''),
+			'$guid' => $app['app_id'] ?? '',
+			'$author' => $app['app_author'] ?? $channel['channel_hash'],
+			'$addr' => $app['app_addr'] ?? $channel['xchan_addr'],
+			'$name' => array('name', t('Name of app'), $app['app_name'] ?? '', t('Required')),
+			'$url' => array('url', t('Location (URL) of app'), $app['app_url'] ?? '', t('Required')),
+	 		'$desc' => array('desc', t('Description'), $app['app_desc'] ?? '', ''),
+			'$photo' => array('photo', t('Photo icon URL'),$app['app_photo'] ?? '', t('80 x 80 pixels - optional')),
+			'$categories' => array('categories',t('Categories (optional, comma separated list)'), $app['categories'] ?? '',''),
+			'$version' => array('version', t('Version ID'), $app['app_version'] ?? '', ''),
+			'$price' => array('price', t('Price of app'), $app['app_price'] ?? '', ''),
+			'$page' => array('page', t('Location (URL) to purchase app'), $app['app_page'] ?? '', ''),
+			'$system' => $app['app_system'] ?? 0,
+			'$plugin' => $app['app_plugin'] ?? '',
+			'$requires' => $app['app_requires'] ?? '',
 			'$embed' => $embed,
 			'$submit' => t('Submit')
 		));
-
 	}
 
 }

Zotlabs/Module/Apps.php

@@ -19,7 +19,8 @@ class Apps extends \Zotlabs\Web\Controller {
 
 		$_SESSION['return_url'] = \App::$query_string;
 
-		$apps = array();
+		$apps = [];
+		$cat = [];
 
 		if(local_channel()) {
 			Zlib\Apps::import_system_apps();

Zotlabs/Module/Article_edit.php

@@ -1,138 +0,0 @@
-<?php
-namespace Zotlabs\Module;
-
-require_once('include/channel.php');
-require_once('include/acl_selectors.php');
-require_once('include/conversation.php');
-
-class Article_edit extends \Zotlabs\Web\Controller {
-
-
-	function get() {
-
-		// Figure out which post we're editing
-		$post_id = ((argc() > 1) ? intval(argv(1)) : 0);
-
-		if(! $post_id) {
-			notice( t('Item not found') . EOL);
-			return;
-		}
-
-		$itm = q("SELECT * FROM item WHERE id = %d and item_type = %d LIMIT 1",
-			intval($post_id),
-			intval(ITEM_TYPE_ARTICLE)
-		);
-		if($itm) {
-			$item_id = q("select * from iconfig where cat = 'system' and k = 'ARTICLE' and iid = %d limit 1",
-				intval($itm[0]['id'])
-			);
-			if($item_id)
-				$card_title = $item_id[0]['v'];
-		}
-		else {
-			notice( t('Item not found') . EOL);
-			return;
-		}
-
-		$owner = $itm[0]['uid'];
-		$uid = local_channel();
-
-		$observer = \App::get_observer();
-
-		$channel = channelx_by_n($owner);
-		if(! $channel) {
-			notice( t('Channel not found.') . EOL);
-			return;
-		}
-
-		$ob_hash = (($observer) ? $observer['xchan_hash'] : '');
-
-		if(! perm_is_allowed($owner,$ob_hash,'write_pages')) {
-			notice( t('Permission denied.') . EOL);
-			return;
-		}
-
-		$is_owner = (($uid && $uid == $owner) ? true : false);
-
-		$o = '';
-
-
-
-		$category = '';
-		$catsenabled = ((feature_enabled($owner,'categories')) ? 'categories' : '');
-
-		if ($catsenabled){
-		        $itm = fetch_post_tags($itm);
-
-	                $cats = get_terms_oftype($itm[0]['term'], TERM_CATEGORY);
-
-		        foreach ($cats as $cat) {
-		                if (strlen($category))
-		                        $category .= ', ';
-		                $category .= $cat['term'];
-		        }
-		}
-
-		if($itm[0]['attach']) {
-			$j = json_decode($itm[0]['attach'],true);
-			if($j) {
-				foreach($j as $jj) {
-					$itm[0]['body'] .= "\n" . '[attachment]' . basename($jj['href']) . ',' . $jj['revision'] . '[/attachment]' . "\n";
-				}
-			}
-		}
-
-
-		$mimetype = $itm[0]['mimetype'];
-
-		$content = $itm[0]['body'];
-
-		$rp = 'articles/' . $channel['channel_address'];
-
-		$x = array(
-			'nickname' => $channel['channel_address'],
-			'bbco_autocomplete'=> 'bbcode',
-			'return_path' => $rp,
-			'webpage' => ITEM_TYPE_ARTICLE,
-			'button' => t('Edit'),
-			'writefiles' => perm_is_allowed($owner, get_observer_hash(), 'write_pages'),
-			'weblink' => t('Insert web link'),
-			'hide_voting' => false,
-			'hide_future' => false,
-			'hide_location' => false,
-			'hide_expire' => false,
-			'showacl' => true,
-			'acl' => populate_acl($itm[0],false,\Zotlabs\Lib\PermissionDescription::fromGlobalPermission('view_pages')),
-			'permissions' => $itm[0],
-			'lockstate' => (($itm[0]['allow_cid'] || $itm[0]['allow_gid'] || $itm[0]['deny_cid'] || $itm[0]['deny_gid']) ? 'lock' : 'unlock'),
-			'ptyp' => $itm[0]['type'],
-			'mimeselect' => false,
-			'mimetype' => $itm[0]['mimetype'],
-			'body' => undo_post_tagging($content),
-			'post_id' => $post_id,
-			'visitor' => true,
-			'title' => htmlspecialchars($itm[0]['title'],ENT_COMPAT,'UTF-8'),
-			'summary' => htmlspecialchars($itm[0]['summary'],ENT_COMPAT,'UTF-8'),
-			'placeholdertitle' => t('Title (optional)'),
-			'pagetitle' => $card_title,
-			'profile_uid' => (intval($channel['channel_id'])),
-			'catsenabled' => $catsenabled,
-			'category' => $category,
-			'bbcode' => (($mimetype  == 'text/bbcode') ? true : false)
-		);
-
-		$editor = status_editor($a, $x, false, 'Article_edit');
-
-		$o .= replace_macros(get_markup_template('edpost_head.tpl'), array(
-			'$title' => t('Edit Article'),
-			'$delete' => ((($itm[0]['author_xchan'] === $ob_hash) || ($itm[0]['owner_xchan'] === $ob_hash)) ? t('Delete') : false),
-			'$id' => $itm[0]['id'],
-			'$cancel' => t('Cancel'),
-			'$editor' => $editor
-		));
-
-		return $o;
-
-	}
-
-}

Zotlabs/Module/Articles.php

@@ -1,232 +0,0 @@
-<?php
-namespace Zotlabs\Module;
-
-use App;
-use Zotlabs\Lib\Apps;
-use Zotlabs\Web\Controller;
-use Zotlabs\Lib\PermissionDescription;
-
-require_once('include/channel.php');
-require_once('include/conversation.php');
-require_once('include/acl_selectors.php');
-require_once('include/opengraph.php');
-
-
-class Articles extends Controller {
-
-	function init() {
-
-		if(argc() > 1)
-			$which = argv(1);
-
-                if(! $which) {
-                        if(local_channel()) {
-                                $channel = App::get_channel();
-                                if($channel && $channel['channel_address'])
-                                $which = $channel['channel_address'];
-                        } else {
-				return;
-			}
-                }
-
-		profile_load($which);
-
-	}
-
-	function get($update = 0, $load = false) {
-
-		if(observer_prohibited(true)) {
-			return login();
-		}
-
-		if(! App::$profile) {
-			notice( t('Requested profile is not available.') . EOL );
-			App::$error = 404;
-			return;
-		}
-
-		if(! Apps::system_app_installed(App::$profile_uid, 'Articles')) {
-			//Do not display any associated widgets at this point
-			App::$pdl = '';
-			$papp = Apps::get_papp('Articles');
-			return Apps::app_render($papp, 'module');
-		}
-
-		nav_set_selected('Articles');
-
-		head_add_link([
-			'rel'   => 'alternate',
-			'type'  => 'application/json+oembed',
-			'href'  => z_root() . '/oep?f=&url=' . urlencode(z_root() . '/' . App::$query_string),
-			'title' => 'oembed'
-		]);
-
-
-		$category = (($_REQUEST['cat']) ? escape_tags(trim($_REQUEST['cat'])) : '');
-
-		if($category) {
-			$sql_extra2 .= protect_sprintf(term_item_parent_query(App::$profile['profile_uid'],'item', $category, TERM_CATEGORY));
-		}
-
-		$datequery = ((x($_GET,'dend') && is_a_date_arg($_GET['dend'])) ? notags($_GET['dend']) : '');
-		$datequery2 = ((x($_GET,'dbegin') && is_a_date_arg($_GET['dbegin'])) ? notags($_GET['dbegin']) : '');
-
-		$which = argv(1);
-
-		$selected_card = ((argc() > 2) ? argv(2) : '');
-
-		$_SESSION['return_url'] = App::$query_string;
-
-		$uid      = local_channel();
-		$owner    = App::$profile_uid;
-		$observer = App::get_observer();
-
-		$ob_hash = (($observer) ? $observer['xchan_hash'] : '');
-
-		if(! perm_is_allowed($owner,$ob_hash,'view_pages')) {
-			notice( t('Permission denied.') . EOL);
-			return;
-		}
-
-		$is_owner = ($uid && $uid == $owner);
-
-		$channel = channelx_by_n($owner);
-
-		if($channel) {
-			$channel_acl = array(
-				'allow_cid' => $channel['channel_allow_cid'],
-				'allow_gid' => $channel['channel_allow_gid'],
-				'deny_cid'  => $channel['channel_deny_cid'],
-				'deny_gid'  => $channel['channel_deny_gid']
-			);
-		}
-		else {
-			$channel_acl = [ 'allow_cid' => '', 'allow_gid' => '', 'deny_cid' => '', 'deny_gid' => '' ];
-		}
-
-
-
-		if(perm_is_allowed($owner,$ob_hash,'write_pages')) {
-
-			$x = [
-				'webpage'           => ITEM_TYPE_ARTICLE,
-				'is_owner'          => true,
-				'content_label'     => t('Add Article'),
-				'button'            => t('Save'),
-				'nickname'          => $channel['channel_address'],
-				'lockstate'         => (($channel['channel_allow_cid'] || $channel['channel_allow_gid']
-					|| $channel['channel_deny_cid'] || $channel['channel_deny_gid']) ? 'lock' : 'unlock'),
-				'acl'               => (($is_owner) ? populate_acl($channel_acl, false,
-					PermissionDescription::fromGlobalPermission('view_pages')) : ''),
-				'permissions'       => $channel_acl,
-				'showacl'           => (($is_owner) ? true : false),
-				'visitor'           => true,
-				'hide_location'     => false,
-				'hide_voting'       => false,
-				'profile_uid'       => intval($owner),
-				'mimetype'          => 'text/bbcode',
-				'mimeselect'        => false,
-				'layoutselect'      => false,
-				'expanded'          => false,
-				'novoting'          => false,
-				'catsenabled'       => feature_enabled($owner,'categories'),
-				'bbco_autocomplete' => 'bbcode',
-				'bbcode'            => true
-			];
-
-			if($_REQUEST['title'])
-				$x['title'] = $_REQUEST['title'];
-			if($_REQUEST['body'])
-				$x['body'] = $_REQUEST['body'];
-			$editor = status_editor($a,$x,false,'Articles');
-
-		}
-		else {
-			$editor = '';
-		}
-
-		$itemspage = get_pconfig(local_channel(),'system','itemspage');
-		App::set_pager_itemspage(((intval($itemspage)) ? $itemspage : 10));
-		$pager_sql = sprintf(" LIMIT %d OFFSET %d ", intval(App::$pager['itemspage']), intval(App::$pager['start']));
-
-
-		$sql_extra = item_permissions_sql($owner);
-		$sql_item = '';
-
-		if($selected_card) {
-			$r = q("select * from iconfig where iconfig.cat = 'system' and iconfig.k = 'ARTICLE' and iconfig.v = '%s' limit 1",
-				dbesc($selected_card)
-			);
-			if($r) {
-				$sql_item = "and item.id = " . intval($r[0]['iid']) . " ";
-			}
-		}
-		if($datequery) {
-			$sql_extra2 .= protect_sprintf(sprintf(" AND item.created <= '%s' ", dbesc(datetime_convert(date_default_timezone_get(),'',$datequery))));
-			$order = 'post';
-		}
-		if($datequery2) {
-			$sql_extra2 .= protect_sprintf(sprintf(" AND item.created >= '%s' ", dbesc(datetime_convert(date_default_timezone_get(),'',$datequery2))));
-		}
-
-		if($datequery || $datequery2) {
-			$sql_extra2 .= " and item.item_thread_top != 0 ";
-		}
-
-		$r = q("select * from item
-			where item.uid = %d and item_type = %d
-			$sql_extra $sql_extra2 $sql_item order by item.created desc $pager_sql",
-			intval($owner),
-			intval(ITEM_TYPE_ARTICLE)
-		);
-
-		$item_normal = " and item.item_hidden = 0 and item.item_type in (0,7) and item.item_deleted = 0
-			and item.item_unpublished = 0 and item.item_delayed = 0 and item.item_pending_remove = 0
-			and item.item_blocked = 0 ";
-
-		if($r) {
-
-			$pager_total = count($r);
-
-			$parents_str = ids_to_querystr($r,'id');
-
-			$r = q("SELECT item.*, item.id AS item_id
-				FROM item
-				WHERE item.uid = %d $item_normal
-				AND item.parent IN ( %s )
-				$sql_extra $sql_extra2 ",
-				intval(App::$profile['profile_uid']),
-				dbesc($parents_str)
-			);
-			if($r) {
-				xchan_query($r);
-				$items = fetch_post_tags($r, true);
-				$items = conv_sort($items,'updated');
-			}
-			else
-				$items = [];
-		}
-
-		// Add Opengraph markup
-		opengraph_add_meta((! empty($items) ? $r[0] : []), $channel);
-
-		$mode = 'articles';
-
-		if(get_pconfig(local_channel(),'system','articles_list_mode') && (! $selected_card))
-            $page_mode = 'pager_list';
-        else
-            $page_mode = 'traditional';
-
-     	$content = conversation($items,$mode,false,$page_mode);
-
-		$o = replace_macros(get_markup_template('cards.tpl'), [
-			'$title' => t('Articles'),
-			'$editor' => $editor,
-			'$content' => $content,
-			'$pager' => alt_pager($pager_total)
-		]);
-
-        return $o;
-    }
-
-}

Zotlabs/Module/Attach.php

@@ -95,7 +95,7 @@ class Attach extends Controller {
 			killme();
 		}
 
-		$r = attach_by_hash(argv(1), get_observer_hash(), ((argc() > 2) ? intval(argv(2)) : 0));
+		$r = attach_by_hash(argv(1), get_observer_hash(), ((argc() > 2) ? intval(argv(2)) : 0), $token);
 
 		if (!$r['success']) {
 			notice($r['message'] . EOL);

Zotlabs/Module/Blocks.php

@@ -9,43 +9,43 @@ require_once('include/acl_selectors.php');
 class Blocks extends \Zotlabs\Web\Controller {
 
 	function init() {
-	
+
 		if(argc() > 1 && argv(1) === 'sys' && is_site_admin()) {
 			$sys = get_sys_channel();
 			if($sys && intval($sys['channel_id'])) {
 				\App::$is_sys = true;
 			}
 		}
-	
+
 		if(argc() > 1)
 			$which = argv(1);
 		else
 			return;
-	
+
 		profile_load($which);
-	
+
 	}
-	
-	
+
+
 	function get() {
-	
+
 		if(! \App::$profile) {
 			notice( t('Requested profile is not available.') . EOL );
 			\App::$error = 404;
 			return;
 		}
-	
+
 		$which = argv(1);
-	
+
 		$_SESSION['return_url'] = \App::$query_string;
-	
+
 		$uid = local_channel();
 		$owner = 0;
 		$channel = null;
 		$observer = \App::get_observer();
-	
+
 		$channel = \App::get_channel();
-	
+
 		if(\App::$is_sys && is_site_admin()) {
 			$sys = get_sys_channel();
 			if($sys && intval($sys['channel_id'])) {
@@ -54,7 +54,7 @@ class Blocks extends \Zotlabs\Web\Controller {
 				$observer = $sys;
 			}
 		}
-	
+
 		if(! $owner) {
 			// Figure out who the page owner is.
 			$r = q("select channel_id from channel where channel_address = '%s'",
@@ -64,24 +64,24 @@ class Blocks extends \Zotlabs\Web\Controller {
 				$owner = intval($r[0]['channel_id']);
 			}
 		}
-	
+
 		$ob_hash = (($observer) ? $observer['xchan_hash'] : '');
-	
+
 		$perms = get_all_perms($owner,$ob_hash);
-	
+
 		if(! $perms['write_pages']) {
 			notice( t('Permission denied.') . EOL);
 			return;
 		}
-	
-		// Block design features from visitors 
-	
+
+		// Block design features from visitors
+
 		if((! $uid) || ($uid != $owner)) {
 			notice( t('Permission denied.') . EOL);
 			return;
 		}
-	
-		$mimetype = (($_REQUEST['mimetype']) ? $_REQUEST['mimetype'] : get_pconfig($owner,'system','page_mimetype'));
+
+		$mimetype = ((isset($_REQUEST['mimetype']) && $_REQUEST['mimetype']) ? $_REQUEST['mimetype'] : get_pconfig($owner,'system','page_mimetype'));
 
 		$x = array(
 			'webpage' => ITEM_TYPE_BLOCK,
@@ -101,27 +101,25 @@ class Blocks extends \Zotlabs\Web\Controller {
 			'bbco_autocomplete' => 'bbcode',
 			'bbcode' => true
 		);
-	
-		if($_REQUEST['title'])
-			$x['title'] = $_REQUEST['title'];
-		if($_REQUEST['body'])
-			$x['body'] = $_REQUEST['body'];
-		if($_REQUEST['pagetitle'])
-			$x['pagetitle'] = $_REQUEST['pagetitle'];
-	
-		$editor = status_editor($a,$x,false,'Blocks');
-	
 
-		$r = q("select iconfig.iid, iconfig.k, iconfig.v, mid, title, body, mimetype, created, edited from iconfig 
+		$x['title'] = $_REQUEST['title'] ?? '';
+		$x['body'] = $_REQUEST['body'] ?? '';
+		$x['pagetitle'] = $_REQUEST['pagetitle'] ?? '';
+
+		$a = '';
+		$editor = status_editor($x, false, 'Blocks');
+
+
+		$r = q("select iconfig.iid, iconfig.k, iconfig.v, mid, title, body, mimetype, created, edited from iconfig
 			left join item on iconfig.iid = item.id
-			where uid = %d and iconfig.cat = 'system' and iconfig.k = 'BUILDBLOCK' 
+			where uid = %d and iconfig.cat = 'system' and iconfig.k = 'BUILDBLOCK'
 			and item_type = %d order by item.created desc",
 			intval($owner),
 			intval(ITEM_TYPE_BLOCK)
 		);
-	
+
 		$pages = null;
-	
+
 		if($r) {
 			$pages = array();
 			foreach($r as $rr) {
@@ -143,13 +141,13 @@ class Blocks extends \Zotlabs\Web\Controller {
 					'edited' => $rr['edited'],
 					'bb_element' => '[element]' . base64url_encode(json_encode($element_arr)) . '[/element]'
 				);
-			} 
+			}
 		}
-	
+
 		//Build the base URL for edit links
-		$url = z_root() . '/editblock/' . $which; 
-	
-		$o .= replace_macros(get_markup_template('blocklist.tpl'), array(
+		$url = z_root() . '/editblock/' . $which;
+
+		$o = replace_macros(get_markup_template('blocklist.tpl'), array(
 			'$baseurl'    => $url,
 			'$title'      => t('Blocks'),
 			'$name'       => t('Block Name'),
@@ -166,8 +164,8 @@ class Blocks extends \Zotlabs\Web\Controller {
 			'$view'       => t('View'),
 			'$preview'    => '1',
 		));
-	    
+
 		return $o;
 	}
-	
+
 }

Zotlabs/Module/Cal.php

@@ -39,7 +39,7 @@ class Cal extends Controller {
 
 			head_set_icon(App::$data['channel']['xchan_photo_s']);
 
-			App::$page['htmlhead'] .= "<script> var profile_uid = " . ((App::$data['channel']) ? App::$data['channel']['channel_id'] : 0) . "; </script>" ;
+			App::$page['htmlhead'] = "<script> var profile_uid = " . ((App::$data['channel']) ? App::$data['channel']['channel_id'] : 0) . "; </script>" ;
 
 		}
 
@@ -65,13 +65,9 @@ class Cal extends Controller {
 
 		nav_set_selected('Calendar');
 
-		head_add_css('/library/fullcalendar/packages/core/main.min.css');
-		head_add_css('/library/fullcalendar/packages/daygrid/main.min.css');
+		head_add_js('/library/fullcalendar/dist/index.global.js');
 		head_add_css('cdav_calendar.css');
 
-		head_add_js('/library/fullcalendar/packages/core/main.min.js');
-		head_add_js('/library/fullcalendar/packages/daygrid/main.min.js');
-
 		$sql_extra = permissions_sql($channel['channel_id'], get_observer_hash(), 'event');
 
 		if(! perm_is_allowed($channel['channel_id'], get_observer_hash(), 'view_contacts') || App::$profile['hide_friends'])

Zotlabs/Module/Card_edit.php

@@ -1,140 +0,0 @@
-<?php
-namespace Zotlabs\Module;
-
-require_once('include/channel.php');
-require_once('include/acl_selectors.php');
-require_once('include/conversation.php');
-
-class Card_edit extends \Zotlabs\Web\Controller {
-
-
-	function get() {
-
-		// Figure out which post we're editing
-		$post_id = ((argc() > 1) ? intval(argv(1)) : 0);
-
-		if(! $post_id) {
-			notice( t('Item not found') . EOL);
-			return;
-		}
-
-		$itm = q("SELECT * FROM item WHERE id = %d and item_type = %d LIMIT 1",
-			intval($post_id),
-			intval(ITEM_TYPE_CARD)
-		);
-		if($itm) {
-			$item_id = q("select * from iconfig where cat = 'system' and k = 'CARD' and iid = %d limit 1",
-				intval($itm[0]['id'])
-			);
-			if($item_id)
-				$card_title = $item_id[0]['v'];
-		}
-		else {
-			notice( t('Item not found') . EOL);
-			return;
-		}
-
-		$owner = $itm[0]['uid'];
-		$uid = local_channel();
-
-		$observer = \App::get_observer();
-
-		$channel = channelx_by_n($owner);
-		if(! $channel) {
-			notice( t('Channel not found.') . EOL);
-			return;
-		}
-
-		$ob_hash = (($observer) ? $observer['xchan_hash'] : '');
-
-		if(! perm_is_allowed($owner,$ob_hash,'write_pages')) {
-			notice( t('Permission denied.') . EOL);
-			return;
-		}
-
-		$is_owner = (($uid && $uid == $owner) ? true : false);
-
-		$o = '';
-
-
-
-		$category = '';
-		$catsenabled = ((feature_enabled($owner,'categories')) ? 'categories' : '');
-
-		if ($catsenabled){
-		        $itm = fetch_post_tags($itm);
-
-	                $cats = get_terms_oftype($itm[0]['term'], TERM_CATEGORY);
-
-		        foreach ($cats as $cat) {
-		                if (strlen($category))
-		                        $category .= ', ';
-		                $category .= $cat['term'];
-		        }
-		}
-
-		if($itm[0]['attach']) {
-			$j = json_decode($itm[0]['attach'],true);
-			if($j) {
-				foreach($j as $jj) {
-					$itm[0]['body'] .= "\n" . '[attachment]' . basename($jj['href']) . ',' . $jj['revision'] . '[/attachment]' . "\n";
-				}
-			}
-		}
-
-
-		$mimetype = $itm[0]['mimetype'];
-
-		$content = $itm[0]['body'];
-
-
-
-		$rp = 'cards/' . $channel['channel_address'];
-
-		$x = array(
-			'nickname' => $channel['channel_address'],
-			'bbco_autocomplete'=> 'bbcode',
-			'return_path' => $rp,
-			'webpage' => ITEM_TYPE_CARD,
-			'button' => t('Edit'),
-			'writefiles' => perm_is_allowed($owner, get_observer_hash(), 'write_pages'),
-			'weblink' => t('Insert web link'),
-			'hide_voting' => false,
-			'hide_future' => false,
-			'hide_location' => false,
-			'hide_expire' => false,
-			'showacl' => true,
-			'acl' => populate_acl($itm[0],false,\Zotlabs\Lib\PermissionDescription::fromGlobalPermission('view_pages')),
-			'permissions' => $itm[0],
-			'lockstate' => (($itm[0]['allow_cid'] || $itm[0]['allow_gid'] || $itm[0]['deny_cid'] || $itm[0]['deny_gid']) ? 'lock' : 'unlock'),
-			'ptyp' => $itm[0]['type'],
-			'mimeselect' => false,
-			'mimetype' => $itm[0]['mimetype'],
-			'body' => undo_post_tagging($content),
-			'post_id' => $post_id,
-			'visitor' => true,
-			'title' => htmlspecialchars($itm[0]['title'],ENT_COMPAT,'UTF-8'),
-			'summary' => htmlspecialchars($itm[0]['summary'],ENT_COMPAT,'UTF-8'),
-			'placeholdertitle' => t('Title (optional)'),
-			'pagetitle' => $card_title,
-			'profile_uid' => (intval($channel['channel_id'])),
-			'catsenabled' => $catsenabled,
-			'category' => $category,
-			'bbcode' => (($mimetype  == 'text/bbcode') ? true : false)
-		);
-
-		$editor = status_editor($a, $x, false, 'Card_edit');
-
-		$o .= replace_macros(get_markup_template('edpost_head.tpl'), array(
-			'$title' => t('Edit Card'),
-			'$delete' => ((($itm[0]['author_xchan'] === $ob_hash) || ($itm[0]['owner_xchan'] === $ob_hash)) ? t('Delete') : false),
-			'$id' => $itm[0]['id'],
-			'$cancel' => t('Cancel'),
-			'$editor' => $editor
-		));
-
-		return $o;
-
-	}
-
-}

Zotlabs/Module/Cards.php

@@ -1,214 +0,0 @@
-<?php
-namespace Zotlabs\Module;
-
-use App;
-use Zotlabs\Lib\Apps;
-use Zotlabs\Web\Controller;
-use Zotlabs\Lib\PermissionDescription;
-
-require_once('include/channel.php');
-require_once('include/conversation.php');
-require_once('include/acl_selectors.php');
-
-/**
- * @brief Provides the Cards module.
- *
- */
-class Cards extends Controller {
-
-	public function init() {
-
-		if(argc() > 1)
-			$which = argv(1);
-		else
-			return;
-
-		profile_load($which);
-	}
-
-	/**
-	 * {@inheritDoc}
-	 * @see \\Zotlabs\\Web\\Controller::get()
-	 *
-	 * @return string Parsed HTML from template 'cards.tpl'
-	 */
-	public function get($update = 0, $load = false) {
-
-		if(observer_prohibited(true)) {
-			return login();
-		}
-
-		if(! App::$profile) {
-			notice( t('Requested profile is not available.') . EOL );
-			App::$error = 404;
-			return;
-		}
-
-		if(! Apps::system_app_installed(App::$profile_uid, 'Cards')) {
-			//Do not display any associated widgets at this point
-			App::$pdl = '';
-			$papp = Apps::get_papp('Cards');
-			return Apps::app_render($papp, 'module');
-		}
-
-		nav_set_selected('Cards');
-
-		head_add_link([
-			'rel'   => 'alternate',
-			'type'  => 'application/json+oembed',
-			'href'  => z_root() . '/oep?f=&url=' . urlencode(z_root() . '/' . App::$query_string),
-			'title' => 'oembed'
-		]);
-
-
-		$category = (($_REQUEST['cat']) ? escape_tags(trim($_REQUEST['cat'])) : '');
-
-		if($category) {
-			$sql_extra2 .= protect_sprintf(term_item_parent_query(App::$profile['profile_uid'], 'item', $category, TERM_CATEGORY));
-		}
-
-
-		$which = argv(1);
-
-		$selected_card = ((argc() > 2) ? argv(2) : '');
-
-		$_SESSION['return_url'] = App::$query_string;
-
-		$uid      = local_channel();
-		$owner    = App::$profile_uid;
-		$observer = App::get_observer();
-
-		$ob_hash = (($observer) ? $observer['xchan_hash'] : '');
-
-		if(! perm_is_allowed($owner, $ob_hash, 'view_pages')) {
-			notice( t('Permission denied.') . EOL);
-			return;
-		}
-
-		$is_owner = ($uid && $uid == $owner);
-
-		$channel = channelx_by_n($owner);
-
-		if($channel) {
-			$channel_acl = [
-				'allow_cid' => $channel['channel_allow_cid'],
-				'allow_gid' => $channel['channel_allow_gid'],
-				'deny_cid'  => $channel['channel_deny_cid'],
-				'deny_gid'  => $channel['channel_deny_gid']
-			];
-		}
-		else {
-			$channel_acl = [ 'allow_cid' => '', 'allow_gid' => '', 'deny_cid' => '', 'deny_gid' => '' ];
-		}
-
-
-		if(perm_is_allowed($owner, $ob_hash, 'write_pages')) {
-
-			$x = [
-				'webpage'           => ITEM_TYPE_CARD,
-				'is_owner'          => true,
-				'content_label'     => t('Add Card'),
-				'button'            => t('Save'),
-				'nickname'          => $channel['channel_address'],
-				'lockstate'         => (($channel['channel_allow_cid'] || $channel['channel_allow_gid']
-					|| $channel['channel_deny_cid'] || $channel['channel_deny_gid']) ? 'lock' : 'unlock'),
-				'acl'               => (($is_owner) ? populate_acl($channel_acl, false,
-					PermissionDescription::fromGlobalPermission('view_pages')) : ''),
-				'permissions'       => $channel_acl,
-				'showacl'           => (($is_owner) ? true : false),
-				'visitor'           => true,
-				'hide_location'     => false,
-				'hide_voting'       => false,
-				'profile_uid'       => intval($owner),
-				'mimetype'          => 'text/bbcode',
-				'mimeselect'        => false,
-				'layoutselect'      => false,
-				'expanded'          => false,
-				'novoting'          => false,
-				'catsenabled'       => feature_enabled($owner, 'categories'),
-				'bbco_autocomplete' => 'bbcode',
-				'bbcode'            => true
-			];
-
-			if($_REQUEST['title'])
-				$x['title'] = $_REQUEST['title'];
-			if($_REQUEST['body'])
-				$x['body'] = $_REQUEST['body'];
-
-			$editor = status_editor($a, $x, false, 'Cards');
-		}
-		else {
-			$editor = '';
-		}
-
-
-		$itemspage = get_pconfig(local_channel(),'system','itemspage');
-		App::set_pager_itemspage(((intval($itemspage)) ? $itemspage : 10));
-		$pager_sql = sprintf(" LIMIT %d OFFSET %d ", intval(App::$pager['itemspage']), intval(App::$pager['start']));
-
-
-		$sql_extra = item_permissions_sql($owner);
-		$sql_item = '';
-
-		if($selected_card) {
-			$r = q("select * from iconfig where iconfig.cat = 'system' and iconfig.k = 'CARD' and iconfig.v = '%s' limit 1",
-				dbesc($selected_card)
-			);
-			if($r) {
-				$sql_item = "and item.id = " . intval($r[0]['iid']) . " ";
-			}
-		}
-
-		$r = q("select * from item
-			where uid = %d and item_type = %d
-			$sql_extra $sql_item order by item.created desc $pager_sql",
-			intval($owner),
-			intval(ITEM_TYPE_CARD)
-		);
-
-		$item_normal = " and item.item_hidden = 0 and item.item_type in (0,6) and item.item_deleted = 0
-			and item.item_unpublished = 0 and item.item_delayed = 0 and item.item_pending_remove = 0
-			and item.item_blocked = 0 ";
-
-		$items_result = [];
-		if($r) {
-
-			$pager_total = count($r);
-
-			$parents_str = ids_to_querystr($r, 'id');
-
-			$items = q("SELECT item.*, item.id AS item_id
-				FROM item
-				WHERE item.uid = %d $item_normal
-				AND item.parent IN ( %s )
-				$sql_extra $sql_extra2 ",
-				intval(App::$profile['profile_uid']),
-				dbesc($parents_str)
-			);
-			if($items) {
-				xchan_query($items);
-				$items = fetch_post_tags($items, true);
-				$items_result = conv_sort($items, 'updated');
-			}
-		}
-
-		$mode = 'cards';
-
-		if(get_pconfig(local_channel(),'system','articles_list_mode') && (! $selected_card))
-			$page_mode = 'pager_list';
-		else
-			$page_mode = 'traditional';
-
-		$content = conversation($items_result, $mode, false, $page_mode);
-
-		$o = replace_macros(get_markup_template('cards.tpl'), [
-			'$title' => t('Cards'),
-			'$editor' => $editor,
-			'$content' => $content,
-			'$pager' => alt_pager($pager_total)
-		]);
-
-		return $o;
-	}
-
-}

Zotlabs/Module/Cdav.php

@@ -200,7 +200,7 @@ class Cdav extends Controller {
 						$etag = (isset($_SERVER['HTTP_IF_MATCH']) ? $_SERVER['HTTP_IF_MATCH'] : false);
 
 						// delete
-						if($httpmethod === 'DELETE' && $cdavdata['etag'] == $etag) {
+						if($httpmethod === 'DELETE' && $etag && isset($cdavdata['etag']) && $cdavdata['etag'] == $etag) {
 							Libsync::build_sync_packet($channel['channel_id'], [
 								$sync => [
 									'action' => 'delete_card',
@@ -210,7 +210,7 @@ class Cdav extends Controller {
 							]);
 						}
 						else {
-							if($etag && $cdavdata['etag'] !== $etag) {
+							if($etag && isset($cdavdata['etag']) && $cdavdata['etag'] !== $etag) {
 								// update
 								Libsync::build_sync_packet($channel['channel_id'], [
 									$sync => [
@@ -317,7 +317,7 @@ class Cdav extends Controller {
 			$calendars = $caldavBackend->getCalendarsForUser($principalUri);
 
 			//create new calendar
-			if($_REQUEST['{DAV:}displayname'] && $_REQUEST['create']) {
+			if((isset($_REQUEST['{DAV:}displayname']) && $_REQUEST['{DAV:}displayname']) && (isset($_REQUEST['create']) && $_REQUEST['create'])) {
 				do {
 					$duplicate = false;
 					$calendarUri = random_string(40);
@@ -332,9 +332,9 @@ class Cdav extends Controller {
 				} while ($duplicate == true);
 
 				$properties = [
-					'{DAV:}displayname' => $_REQUEST['{DAV:}displayname'],
-					'{http://apple.com/ns/ical/}calendar-color' => $_REQUEST['color'],
-					'{urn:ietf:params:xml:ns:caldav}calendar-description' => $channel['channel_name']
+					'{DAV:}displayname' => escape_tags($_REQUEST['{DAV:}displayname']),
+					'{http://apple.com/ns/ical/}calendar-color' => escape_tags($_REQUEST['color']),
+					'{urn:ietf:params:xml:ns:caldav}calendar-description' => escape_tags($channel['channel_name'])
 				];
 
 				$id = $caldavBackend->createCalendar($principalUri, $calendarUri, $properties);
@@ -352,7 +352,7 @@ class Cdav extends Controller {
 			}
 
 			//create new calendar object via ajax request
-			if($_REQUEST['submit'] === 'create_event' && $_REQUEST['title'] && $_REQUEST['target'] && $_REQUEST['dtstart']) {
+			if((isset($_REQUEST['submit']) && $_REQUEST['submit'] === 'create_event') && $_REQUEST['title'] && $_REQUEST['target'] && $_REQUEST['dtstart']) {
 
 				$id = explode(':', $_REQUEST['target']);
 
@@ -366,7 +366,7 @@ class Cdav extends Controller {
 
 				$allday = $_REQUEST['allday'];
 
-				$title = $_REQUEST['title'];
+				$title = escape_tags($_REQUEST['title']);
 				$start = datetime_convert('UTC', 'UTC', $_REQUEST['dtstart']);
 				$dtstart = new \DateTime($start);
 
@@ -374,8 +374,8 @@ class Cdav extends Controller {
 					$end = datetime_convert('UTC', 'UTC', $_REQUEST['dtend']);
 					$dtend = new \DateTime($end);
 				}
-				$description = $_REQUEST['description'];
-				$location = $_REQUEST['location'];
+				$description = escape_tags($_REQUEST['description']);
+				$location = escape_tags($_REQUEST['location']);
 
 				do {
 					$duplicate = false;
@@ -431,7 +431,7 @@ class Cdav extends Controller {
 			}
 
 			//edit calendar name and color
-			if($_REQUEST['{DAV:}displayname'] && $_REQUEST['edit'] && $_REQUEST['id']) {
+			if((isset($_REQUEST['{DAV:}displayname']) && $_REQUEST['{DAV:}displayname']) && $_REQUEST['edit'] && $_REQUEST['id']) {
 
 				$id = explode(':', $_REQUEST['id']);
 
@@ -441,8 +441,8 @@ class Cdav extends Controller {
 				$cdavdata = $this->get_cdav_data($id[0], 'calendarinstances');
 
 				$mutations = [
-					'{DAV:}displayname' => $_REQUEST['{DAV:}displayname'],
-					'{http://apple.com/ns/ical/}calendar-color' => $_REQUEST['color']
+					'{DAV:}displayname' => escape_tags($_REQUEST['{DAV:}displayname']),
+					'{http://apple.com/ns/ical/}calendar-color' => escape_tags($_REQUEST['color'])
 				];
 
 				$patch = new \Sabre\DAV\PropPatch($mutations);
@@ -459,7 +459,7 @@ class Cdav extends Controller {
 			}
 
 			//edit calendar object via ajax request
-			if($_REQUEST['submit'] === 'update_event' && $_REQUEST['uri'] && $_REQUEST['title'] && $_REQUEST['target'] && $_REQUEST['dtstart']) {
+			if((isset($_REQUEST['submit']) && $_REQUEST['submit'] === 'update_event') && $_REQUEST['uri'] && $_REQUEST['title'] && $_REQUEST['target'] && $_REQUEST['dtstart']) {
 
 				$id = explode(':', $_REQUEST['target']);
 
@@ -471,18 +471,18 @@ class Cdav extends Controller {
 				$timezone = ((x($_POST,'timezone_select')) ? escape_tags(trim($_POST['timezone_select'])) : '');
 				$tz = (($timezone) ? $timezone : date_default_timezone_get());
 
-				$allday = $_REQUEST['allday'];
+				$allday = intval($_REQUEST['allday']);
 
-				$uri = $_REQUEST['uri'];
-				$title = $_REQUEST['title'];
+				$uri = escape_tags($_REQUEST['uri']);
+				$title = escape_tags($_REQUEST['title']);
 				$start = datetime_convert('UTC', 'UTC', $_REQUEST['dtstart']);
 				$dtstart = new \DateTime($start);
 				if($_REQUEST['dtend']) {
 					$end = datetime_convert('UTC', 'UTC', $_REQUEST['dtend']);
 					$dtend = new \DateTime($end);
 				}
-				$description = $_REQUEST['description'];
-				$location = $_REQUEST['location'];
+				$description = escape_tags($_REQUEST['description']);
+				$location = escape_tags($_REQUEST['location']);
 
 				$object = $caldavBackend->getCalendarObject($id, $uri);
 
@@ -654,7 +654,7 @@ class Cdav extends Controller {
 						$duplicate = true;
 				} while ($duplicate == true);
 
-				$properties = ['{DAV:}displayname' => $_REQUEST['{DAV:}displayname']];
+				$properties = ['{DAV:}displayname' => escape_tags($_REQUEST['{DAV:}displayname'])];
 
 				$carddavBackend->createAddressBook($principalUri, $addressbookUri, $properties);
 
@@ -668,9 +668,9 @@ class Cdav extends Controller {
 			}
 
 			//edit addressbook
-			if($_REQUEST['{DAV:}displayname'] && $_REQUEST['edit'] && intval($_REQUEST['id'])) {
+			if($_REQUEST['{DAV:}displayname'] && $_REQUEST['edit'] && $_REQUEST['id']) {
 
-				$id = $_REQUEST['id'];
+				$id = intval($_REQUEST['id']);
 
 				if(! cdav_perms($id,$addressbooks))
 					return;
@@ -678,7 +678,7 @@ class Cdav extends Controller {
 				$cdavdata = $this->get_cdav_data($id, 'addressbooks');
 
 				$mutations = [
-					'{DAV:}displayname' => $_REQUEST['{DAV:}displayname']
+					'{DAV:}displayname' => escape_tags($_REQUEST['{DAV:}displayname'])
 				];
 
 				$patch = new \Sabre\DAV\PropPatch($mutations);
@@ -901,19 +901,10 @@ class Cdav extends Controller {
 
 		//Display calendar(s) here
 		if(argc() <= 3 && argv(1) === 'calendar') {
-
-			head_add_css('/library/fullcalendar/packages/core/main.min.css');
-			head_add_css('/library/fullcalendar/packages/daygrid/main.min.css');
-			head_add_css('/library/fullcalendar/packages/timegrid/main.min.css');
-			head_add_css('/library/fullcalendar/packages/list/main.min.css');
+			head_add_js('/library/fullcalendar/dist/index.global.js');
 			head_add_css('cdav_calendar.css');
 
-			head_add_js('/library/fullcalendar/packages/core/main.min.js');
-			head_add_js('/library/fullcalendar/packages/interaction/main.min.js');
-			head_add_js('/library/fullcalendar/packages/daygrid/main.min.js');
-			head_add_js('/library/fullcalendar/packages/timegrid/main.min.js');
-			head_add_js('/library/fullcalendar/packages/list/main.min.js');
-
+			$o = '';
 			$sources = '';
 			$resource_id = '';
 			$resource = null;
@@ -921,6 +912,8 @@ class Cdav extends Controller {
 			if(argc() == 3)
 				$resource_id = argv(2);
 
+			$categories = '';
+
 			if($resource_id) {
 				$r = q("SELECT event.*, item.author_xchan, item.owner_xchan, item.plink, item.id as item_id FROM event LEFT JOIN item ON event.event_hash = item.resource_id
 					WHERE event.uid = %d AND event.event_hash = '%s' LIMIT 1",
@@ -944,7 +937,6 @@ class Cdav extends Controller {
 					$resource = $r[0];
 
 					$catsenabled = feature_enabled(local_channel(),'categories');
-					$categories = '';
 					if($catsenabled){
 						if($r[0]['term']) {
 							$cats = get_terms_oftype($r[0]['term'], TERM_CATEGORY);
@@ -1095,7 +1087,11 @@ class Cdav extends Controller {
 
 			$filters['name'] = 'VCALENDAR';
 			$filters['prop-filters'][0]['name'] = 'VEVENT';
+			$filters['prop-filters'][0]['is-not-defined'] = null;
+			$filters['prop-filters'][0]['param-filters'] = null;
+			$filters['prop-filters'][0]['text-match'] = null;
 			$filters['comp-filters'][0]['name'] = 'VEVENT';
+			$filters['comp-filters'][0]['is-not-defined'] = null;
 			$filters['comp-filters'][0]['time-range']['start'] = $start;
 			$filters['comp-filters'][0]['time-range']['end'] = $end;
 

Zotlabs/Module/Channel.php

@@ -31,8 +31,10 @@ class Channel extends Controller {
 			goaway(z_root() . '/search?f=&search=' . $_GET['search']);
 
 		$which = null;
-		if (argc() > 1)
+		if (argc() > 1) {
 			$which = argv(1);
+		}
+
 		if (!$which) {
 			if (local_channel()) {
 				$channel = App::get_channel();
@@ -40,9 +42,9 @@ class Channel extends Controller {
 					$which = $channel['channel_address'];
 			}
 		}
+
 		if (!$which) {
-			notice(t('You must be logged in to see this page.') . EOL);
-			return;
+			http_status_exit(404, 'Not found');
 		}
 
 		$profile = 0;
@@ -67,7 +69,7 @@ class Channel extends Controller {
 
 			if ($sigdata && $sigdata['signer'] && $sigdata['header_valid']) {
 				$data = json_encode(Libzot::zotinfo(['guid_hash' => $channel['channel_hash'], 'target_url' => $sigdata['signer']]));
-				$s = q("select site_crypto, hubloc_sitekey from site left join hubloc on hubloc_url = site_url where hubloc_id_url = '%s' and hubloc_network = 'zot6' limit 1",
+				$s = q("select site_crypto, hubloc_sitekey from site left join hubloc on hubloc_url = site_url where hubloc_id_url = '%s' and hubloc_network = 'zot6' order by hubloc_id desc limit 1",
 					dbesc($sigdata['signer'])
 				);
 
@@ -245,9 +247,7 @@ class Channel extends Controller {
 				$channel_acl = ['allow_cid' => '', 'allow_gid' => '', 'deny_cid' => '', 'deny_gid' => ''];
 			}
 
-
 			if ($perms['post_wall']) {
-
 				$x = [
 					'is_owner'            => $is_owner,
 					'allow_location'      => ((($is_owner || $observer) && (intval(get_pconfig(App::$profile['profile_uid'], 'system', 'use_browser_location')))) ? true : false),
@@ -267,9 +267,16 @@ class Channel extends Controller {
 					'reset'               => t('Reset form')
 				];
 
-				$o .= status_editor($a, $x, false, 'Channel');
+				$a = '';
+				$o .= status_editor($x, false, 'Channel');
 			}
 
+			// Add pinned content
+			if (!x($_REQUEST, 'mid') && !$search) {
+				$pinned = new \Zotlabs\Widget\Pinned;
+				$r = $pinned->widget(intval(App::$profile['profile_uid']), [ITEM_TYPE_POST]);
+				$o .= $r['html'];
+			}
 		}
 
 
@@ -277,13 +284,9 @@ class Channel extends Controller {
 		 * Get permissions SQL - if $remote_contact is true, our remote user has been pre-verified and we already have fetched his/her groups
 		 */
 
-		$item_normal = " and item.item_hidden = 0 and item.item_type = 0 and item.item_deleted = 0
-		    and item.item_unpublished = 0 and item.item_pending_remove = 0
-		    and item.item_blocked = 0 ";
-		if (!$is_owner)
-			$item_normal .= "and item.item_delayed = 0 ";
+		$item_normal = item_normal();
 		$item_normal_update = item_normal_update();
-		$sql_extra          = item_permissions_sql(App::$profile['profile_uid']);
+		$sql_extra = item_permissions_sql(App::$profile['profile_uid']);
 
 		if (feature_enabled(App::$profile['profile_uid'], 'channel_list_mode') && (!$mid))
 			$page_mode = 'list';
@@ -293,7 +296,7 @@ class Channel extends Controller {
 		$abook_uids = " and abook.abook_channel = " . intval(App::$profile['profile_uid']) . " ";
 
 		$simple_update = '';
-		if ($update && $_SESSION['loadtime'])
+		if ($update && isset($_SESSION['loadtime']))
 			$simple_update = " AND (( item_unseen = 1 AND item.changed > '" . datetime_convert('UTC', 'UTC', $_SESSION['loadtime']) . "' )  OR item.changed > '" . datetime_convert('UTC', 'UTC', $_SESSION['loadtime']) . "' ) ";
 
 		if ($search) {
@@ -420,17 +423,14 @@ class Channel extends Controller {
 			$items = [];
 		}
 
-		// Add pinned content
-		if (!x($_REQUEST, 'mid') && !$search) {
-			$pinned = new \Zotlabs\Widget\Pinned;
-			$r      = $pinned->widget(intval(App::$profile['profile_uid']), [ITEM_TYPE_POST]);
-			$o      .= $r['html'];
-		}
+
 
 		$mode = (($search) ? 'search' : 'channel');
 
 		if ((!$update) && (!$load)) {
 
+
+
 			//if we got a decoded hash we must encode it again before handing to javascript
 			$mid = gen_link_id($mid);
 

Zotlabs/Module/Channel_calendar.php

@@ -30,7 +30,7 @@ class Channel_calendar extends Controller {
 
 		$xchan = ((x($_POST, 'xchan')) ? dbesc($_POST['xchan']) : '');
 
-		// only allow editing your own events. 
+		// only allow editing your own events.
 		if (($xchan) && ($xchan !== get_observer_hash()))
 			return;
 
@@ -55,8 +55,8 @@ class Channel_calendar extends Controller {
 
 		// Don't allow the event to finish before it begins.
 		// It won't hurt anything, but somebody will file a bug report
-		// and we'll waste a bunch of time responding to it. Time that 
-		// could've been spent doing something else. 
+		// and we'll waste a bunch of time responding to it. Time that
+		// could've been spent doing something else.
 
 		if (strcmp($finish, $start) < 0) {
 			notice(t('Event can not end before it has started.') . EOL);
@@ -319,12 +319,12 @@ class Channel_calendar extends Controller {
 				// fixed an issue with "nofinish" events not showing up in the calendar.
 				// There's still an issue if the finish date crosses the end of month.
 				// Noting this for now - it will need to be fixed here and in Friendica.
-				// Ultimately the finish date shouldn't be involved in the query. 
+				// Ultimately the finish date shouldn't be involved in the query.
 
 				$r = q("SELECT event.*, item.plink, item.item_flags, item.author_xchan, item.owner_xchan, item.id as item_id
-					from event left join item on event.event_hash = item.resource_id 
-					where item.resource_type = 'event' and event.uid = %d and event.uid = item.uid $ignored 
-					AND (( event.adjust = 0 AND ( event.dtend >= '%s' or event.nofinish = 1 ) AND event.dtstart <= '%s' ) 
+					from event left join item on event.event_hash = item.resource_id
+					where item.resource_type = 'event' and event.uid = %d and event.uid = item.uid $ignored
+					AND (( event.adjust = 0 AND ( event.dtend >= '%s' or event.nofinish = 1 ) AND event.dtstart <= '%s' )
 					OR  (  event.adjust = 1 AND ( event.dtend >= '%s' or event.nofinish = 1 ) AND event.dtstart <= '%s' )) ",
 					intval(local_channel()),
 					dbesc($start),
@@ -357,7 +357,7 @@ class Channel_calendar extends Controller {
 					$catsenabled = feature_enabled(local_channel(), 'categories');
 					$categories  = '';
 					if ($catsenabled) {
-						if ($rr['term']) {
+						if (isset($rr['term']) && $rr['term']) {
 							$cats = get_terms_oftype($rr['term'], TERM_CATEGORY);
 							foreach ($cats as $cat) {
 								if (strlen($categories))
@@ -449,7 +449,7 @@ class Channel_calendar extends Controller {
 						}
 
 						// The site admin can delete any post/item on the site.
-						// If the item originated on this site+channel the deletion will propagate downstream. 
+						// If the item originated on this site+channel the deletion will propagate downstream.
 						// Otherwise just the local copy is removed.
 
 						if (is_site_admin()) {

Zotlabs/Module/Chanview.php

@@ -16,17 +16,17 @@ class Chanview extends \Zotlabs\Web\Controller {
 
 		$r = null;
 
-		if($_REQUEST['hash']) {
+		if(!empty($_REQUEST['hash'])) {
 			$r = q("select * from xchan where xchan_hash = '%s' and xchan_deleted = 0",
 				dbesc($_REQUEST['hash'])
 			);
 		}
-		if($_REQUEST['address']) {
+		if(!empty($_REQUEST['address'])) {
 			$r = q("select * from xchan where xchan_addr = '%s' and xchan_deleted = 0",
 				dbesc(punify($_REQUEST['address']))
 			);
 		}
-		elseif(local_channel() && intval($_REQUEST['cid'])) {
+		elseif(local_channel() && !empty($_REQUEST['cid'])) {
 			$r = q("SELECT abook.*, xchan.*
 				FROM abook left join xchan on abook_xchan = xchan_hash
 				WHERE abook_channel = %d and abook_id = %d and xchan_deleted = 0",
@@ -34,8 +34,7 @@ class Chanview extends \Zotlabs\Web\Controller {
 				intval($_REQUEST['cid'])
 			);
 		}
-		elseif($_REQUEST['url']) {
-
+		elseif(!empty($_REQUEST['url'])) {
 			// if somebody re-installed they will have more than one xchan, use the most recent name date as this is
 			// the most useful consistently ascending table item we have.
 
@@ -56,7 +55,7 @@ class Chanview extends \Zotlabs\Web\Controller {
 		if(! App::$poi) {
 			logger('mod_chanview: fallback');
 
-			if($_REQUEST['address']) {
+			if(!empty($_REQUEST['address'])) {
 				$href = Webfinger::zot_url(punify($_REQUEST['address']));
 				if($href) {
 					$_REQUEST['url'] = $href;
@@ -65,9 +64,9 @@ class Chanview extends \Zotlabs\Web\Controller {
 
 			$r = null;
 
-			if($_REQUEST['url']) {
+			if(!empty($_REQUEST['url'])) {
 
-				$zf = Zotfinger::exec($_REQUEST['url'], null);
+				$zf = Zotfinger::exec($_REQUEST['url']);
 
 				if(array_path_exists('signature/signer',$zf) && $zf['signature']['signer'] === $_REQUEST['url'] && intval($zf['signature']['header_valid'])) {
 					Libzot::import_xchan($zf['data']);
@@ -79,6 +78,7 @@ class Chanview extends \Zotlabs\Web\Controller {
 					}
 				}
 				if(! $r) {
+
 					if(discover_by_webbie($_REQUEST['url'])) {
 						$r = q("select * from xchan where xchan_url = '%s' and xchan_deleted = 0",
 							dbesc($_REQUEST['url'])

Zotlabs/Module/Chat.php

@@ -174,13 +174,14 @@ class Chat extends Controller {
 			$x = Chatroom::enter($observer,$room_id,'online',$_SERVER['REMOTE_ADDR']);
 			if(! $x)
 				return;
-			$x = q("select * from chatroom where cr_id = %d and cr_uid = %d $sql_extra limit 1",
+
+			$x = q("select * from chatroom where cr_id = %d and cr_uid = %d limit 1",
 				intval($room_id),
 				intval(App::$profile['profile_uid'])
 			);
 
 			if($x) {
-				$acl = new AccessList(false);
+				$acl = new AccessList([]);
 				$acl->set($x[0]);
 
 				$private = $acl->is_private();
@@ -211,7 +212,7 @@ class Chat extends Controller {
 				'$online' => t('I am online'),
 				'$bookmark_link' => $bookmark_link,
 				'$bookmark' => t('Bookmark this room'),
-				'$feature_encrypt' => ((feature_enabled(local_channel(),'content_encrypt')) ? true : false),
+				'$feature_encrypt' => ((feature_enabled(App::$profile['profile_uid'], 'content_encrypt')) ? true : false),
 				'$cipher' => $cipher,
 				'$linkurl' => t('Please enter a link URL:'),
 				'$encrypt' => t('Encrypt text'),
@@ -224,7 +225,7 @@ class Chat extends Controller {
 
 		$o = '';
 
-		$acl = new AccessList($channel);
+		$acl = new AccessList($channel ?? []);
 		$channel_acl = $acl->get();
 
 		$lockstate = (($channel_acl['allow_cid'] || $channel_acl['allow_gid'] || $channel_acl['deny_cid'] || $channel_acl['deny_gid']) ? 'lock' : 'unlock');

Zotlabs/Module/Chatsvc.php

@@ -1,6 +1,6 @@
 <?php /** @file */
 
-namespace Zotlabs\Module; 
+namespace Zotlabs\Module;
 
 require_once('include/security.php');
 
@@ -9,76 +9,76 @@ use \Zotlabs\Lib as Zlib;
 class Chatsvc extends \Zotlabs\Web\Controller {
 
 	function init() {
-	
+
 		//logger('chatsvc');
-	
+
 		$ret = array('success' => false);
-	
+
 		\App::$data['chat']['room_id'] = intval($_REQUEST['room_id']);
 		$x = q("select cr_uid from chatroom where cr_id = %d and cr_id != 0 limit 1",
 			intval(\App::$data['chat']['room_id'])
 		);
 		if(! $x)
 			json_return_and_die($ret);
-	
+
 		\App::$data['chat']['uid'] = $x[0]['cr_uid'];
-	
+
 		if(! perm_is_allowed(\App::$data['chat']['uid'],get_observer_hash(),'chat')) {
 	        json_return_and_die($ret);
 	    }
-	
+
 	}
-	
+
 	function post() {
-	
+
 		$ret = array('success' => false);
-	
+
 		$room_id = \App::$data['chat']['room_id'];
 		$text = escape_tags($_REQUEST['chat_text']);
 		if(! $text)
 			return;
-	
+
 		$sql_extra = permissions_sql(\App::$data['chat']['uid']);
-	
+
 		$r = q("select * from chatroom where cr_uid = %d and cr_id = %d $sql_extra",
 			intval(\App::$data['chat']['uid']),
 			intval(\App::$data['chat']['room_id'])
 		);
 		if(! $r)
 			json_return_and_die($ret);
-	
+
 		$arr = array(
 			'chat_room' => \App::$data['chat']['room_id'],
 			'chat_xchan' => get_observer_hash(),
 			'chat_text' => $text
 		);
-	
+
 		call_hooks('chat_post',$arr);
-	
+
 		$x = q("insert into chat ( chat_room, chat_xchan, created, chat_text )
 			values( %d, '%s', '%s', '%s' )",
 			intval(\App::$data['chat']['room_id']),
 			dbesc(get_observer_hash()),
 			dbesc(datetime_convert()),
-			dbesc(str_rot47(base64url_encode($arr['chat_text'])))		
+			dbesc(str_rot47(base64url_encode($arr['chat_text'])))
 		);
-	
+
 		$ret['success'] = true;
 		json_return_and_die($ret);
 	}
-	
+
 	function get() {
-	
-		$status = strip_tags($_REQUEST['status']);
+
+		$status = ((isset($_REQUEST['status'])) ? strip_tags($_REQUEST['status']) : '');
 		$room_id = intval(\App::$data['chat']['room_id']);
 		$stopped = ((x($_REQUEST,'stopped') && intval($_REQUEST['stopped'])) ? true : false);
-	
+
 		if($status && $room_id) {
-	
+
 			$x = q("select channel_address from channel where channel_id = %d limit 1",
 				intval(\App::$data['chat']['uid'])
-			);			
-	
+			);
+
 			$r = q("update chatpresence set cp_status = '%s', cp_last = '%s' where cp_room = %d and cp_xchan = '%s' and cp_client = '%s'",
 				dbesc($status),
 				dbesc(datetime_convert()),
@@ -86,27 +86,27 @@ class Chatsvc extends \Zotlabs\Web\Controller {
 				dbesc(get_observer_hash()),
 				dbesc($_SERVER['REMOTE_ADDR'])
 			);
-	
-			goaway(z_root() . '/chat/' . $x[0]['channel_address'] . '/' . $room_id);		
+
+			goaway(z_root() . '/chat/' . $x[0]['channel_address'] . '/' . $room_id);
 		}
-	
+
 		if(! $stopped) {
-	
+
 			$lastseen = intval($_REQUEST['last']);
-	
+
 			$ret = array('success' => false);
-	
+
 			$sql_extra = permissions_sql(\App::$data['chat']['uid']);
-	
+
 			$r = q("select * from chatroom where cr_uid = %d and cr_id = %d $sql_extra",
 				intval(\App::$data['chat']['uid']),
 				intval(\App::$data['chat']['room_id'])
 			);
 			if(! $r)
 				json_return_and_die($ret);
-	
+
 			$inroom = array();
-	
+
 			$r = q("select * from chatpresence left join xchan on xchan_hash = cp_xchan where cp_room = %d order by xchan_name",
 				intval(\App::$data['chat']['room_id'])
 			);
@@ -120,9 +120,9 @@ class Chatsvc extends \Zotlabs\Web\Controller {
 						$rv['xchan_url'] = z_root();
 						$rv['xchan_hidden'] = 1;
 						$rv['xchan_photo_mimetype'] = 'image/png';
-						$rv['xchan_photo_l'] = z_root() . '/' . get_default_profile_photo(300); 
-						$rv['xchan_photo_m'] = z_root() . '/' . get_default_profile_photo(80); 
-						$rv['xchan_photo_s'] = z_root() . '/' . get_default_profile_photo(48); 
+						$rv['xchan_photo_l'] = z_root() . '/' . get_default_profile_photo(300);
+						$rv['xchan_photo_m'] = z_root() . '/' . get_default_profile_photo(80);
+						$rv['xchan_photo_s'] = z_root() . '/' . get_default_profile_photo(48);
 
 					}
 
@@ -137,13 +137,13 @@ class Chatsvc extends \Zotlabs\Web\Controller {
 							$status_class = 'online';
 							break;
 					}
-		
+
 					$inroom[] = array('img' => zid($rv['xchan_photo_m']), 'img_type' => $rv['xchan_photo_mimetype'],'name' => $rv['xchan_name'], 'status' => $status, 'status_class' => $status_class);
 				}
 			}
-	
+
 			$chats = array();
-	
+
 			$r = q("select * from chat left join xchan on chat_xchan = xchan_hash where chat_room = %d and chat_id > %d order by created",
 				intval(\App::$data['chat']['room_id']),
 				intval($lastseen)
@@ -152,7 +152,7 @@ class Chatsvc extends \Zotlabs\Web\Controller {
 				foreach($r as $rr) {
 					$chats[] = array(
 						'id' => $rr['chat_id'],
-						'img' => zid($rr['xchan_photo_m']), 
+						'img' => zid($rr['xchan_photo_m']),
 						'img_type' => $rr['xchan_photo_mimetype'],
 						'name' => $rr['xchan_name'],
 						'isotime' => datetime_convert('UTC', date_default_timezone_get(), $rr['created'], 'c'),
@@ -163,22 +163,22 @@ class Chatsvc extends \Zotlabs\Web\Controller {
 				}
 			}
 		}
-	
+
 		$r = q("update chatpresence set cp_last = '%s' where cp_room = %d and cp_xchan = '%s' and cp_client = '%s'",
 			dbesc(datetime_convert()),
 			intval(\App::$data['chat']['room_id']),
 			dbesc(get_observer_hash()),
 			dbesc($_SERVER['REMOTE_ADDR'])
 		);
-	
+
 		$ret['success'] = true;
 		if(! $stopped) {
 			$ret['inroom'] = $inroom;
 			$ret['chats'] = $chats;
 		}
 		json_return_and_die($ret);
-	
+
 	}
-			 
-	
+
+
 }

Zotlabs/Module/Cloud.php

@@ -70,7 +70,7 @@ class Cloud extends Controller {
 			$_SESSION['cloud_sort'] = 'name';
 		}
 
-		$_SESSION['cloud_sort'] = (($_REQUEST['sort']) ? trim(notags($_REQUEST['sort'])) : $_SESSION['cloud_sort']);
+		$_SESSION['cloud_sort'] = ((isset($_REQUEST['sort']) && $_REQUEST['sort']) ? trim(notags($_REQUEST['sort'])) : $_SESSION['cloud_sort']);
 
 		$x = clean_query_string();
 		if($x !== \App::$query_string)
@@ -100,7 +100,6 @@ class Cloud extends Controller {
 
 
 		// over-ride the default XML output on thrown exceptions
-
 		$server->on('exception', [ $this, 'DAVException' ]);
 
 		// All we need to do now, is to fire up the server
@@ -117,21 +116,19 @@ class Cloud extends Controller {
 	function DAVException($err) {
 
 		if($err instanceof \Sabre\DAV\Exception\NotFound) {
-			notice( t('Not found') . EOL);
+			\App::$page['content'] = '<h2>404 Not found</h2>';
 		}
 		elseif($err instanceof \Sabre\DAV\Exception\Forbidden) {
-			notice( t('Permission denied') . EOL);
+			\App::$page['content'] = '<h2>403 Forbidden</h2>';
 		}
 		elseif($err instanceof \Sabre\DAV\Exception\NotImplemented) {
-			// notice( t('Please refresh page') . EOL);
 			goaway(z_root() . '/' . \App::$query_string);
 		}
 		else {
-			notice( t('Unknown error') . EOL);
+			\App::$page['content'] = '<h2>Unknown error</h2>';
 		}
 
 		construct_page();
-
 		killme();
 	}
 

Zotlabs/Module/Cloud_tiles.php

@@ -6,7 +6,7 @@ class Cloud_tiles extends \Zotlabs\Web\Controller {
 
 	function init() {
 
-		if(intval($_SESSION['cloud_tiles']))
+		if(isset($_SESSION['cloud_tiles']) && intval($_SESSION['cloud_tiles']))
 			$_SESSION['cloud_tiles'] = 0;
 		else
 			$_SESSION['cloud_tiles'] = 1;
@@ -18,4 +18,4 @@ class Cloud_tiles extends \Zotlabs\Web\Controller {
 		goaway(z_root() . '/' . hex2bin(argv(1)));
 
 	}
-}
+}

Zotlabs/Module/Connections.php

@@ -45,7 +45,7 @@ class Connections extends \Zotlabs\Web\Controller {
 		$unconnected = false;
 		$all         = false;
 
-		if(! $_REQUEST['aj'])
+		if(!(isset($_REQUEST['aj']) && $_REQUEST['aj']))
 			$_SESSION['return_url'] = App::$query_string;
 
 		$search_flags = "";
@@ -128,7 +128,9 @@ class Connections extends \Zotlabs\Web\Controller {
 			$unblocked = true;
 		}
 
-		switch($_REQUEST['order']) {
+		$order = $_REQUEST['order'] ?? '';
+
+		switch($order) {
 			case 'name_desc':
 				$sql_order = 'xchan_name DESC';
 				break;
@@ -143,6 +145,7 @@ class Connections extends \Zotlabs\Web\Controller {
 		}
 
 		$search = ((x($_REQUEST,'search')) ? notags(trim($_REQUEST['search'])) : '');
+		$search_xchan = ((x($_REQUEST,'search_xchan')) ? notags(trim($_REQUEST['search_xchan'])) : '');
 
 		$tabs = array(
 			/*
@@ -227,14 +230,21 @@ class Connections extends \Zotlabs\Web\Controller {
 		//$t = replace_macros($tab_tpl, array('$tabs'=>$tabs));
 
 		$searching = false;
+		$search_hdr = '';
+
 		if($search) {
 			$search_hdr = $search;
-			$search_txt = dbesc(protect_sprintf(preg_quote($search)));
-			$searching = true;
-		}
-		$sql_extra .= (($searching) ? protect_sprintf(" AND xchan_name like '%$search_txt%' ") : "");
+			$search_txt = (($search_xchan) ? urldecode($search_xchan) : preg_quote($search));
 
-		if($_REQUEST['gid']) {
+			if ($search_xchan) {
+				$sql_extra .= " AND xchan_hash = '" . protect_sprintf(dbesc($search_txt)) . "' ";
+			}
+			else {
+				$sql_extra .= " AND xchan_name LIKE '%%" . protect_sprintf(dbesc($search_txt)) . "%%' ";
+			}
+		}
+
+		if(isset($_REQUEST['gid']) && $_REQUEST['gid']) {
 			$sql_extra .= " and xchan_hash in ( select xchan from pgrp_member where gid = " . intval($_REQUEST['gid']) . " and uid = " . intval(local_channel()) . " ) ";
 		}
 
@@ -272,14 +282,13 @@ class Connections extends \Zotlabs\Web\Controller {
 			foreach($r as $rr) {
 				if($rr['xchan_url']) {
 
-					if(($rr['vcard']) && is_array($rr['vcard']['tels']) && $rr['vcard']['tels'][0]['nr'])
+					if((isset($rr['vcard'])) && is_array($rr['vcard']['tels']) && $rr['vcard']['tels'][0]['nr'])
 						$phone = $rr['vcard']['tels'][0]['nr'];
 					else
 						$phone = '';
 
 					$status_str = '';
 					$status = array(
-						((intval($rr['abook_active'])) ? t('Active') : ''),
 						((intval($rr['abook_pending'])) ? t('Pending approval') : ''),
 						((intval($rr['abook_archived'])) ? t('Archived') : ''),
 						((intval($rr['abook_hidden'])) ? t('Hidden') : ''),
@@ -354,9 +363,9 @@ class Connections extends \Zotlabs\Web\Controller {
 						'oneway' => $oneway,
 						'perminfo' => $perminfo,
 						'connect' => (intval($rr['abook_not_here']) ? t('Connect') : ''),
-						'follow' => z_root() . '/follow/?f=&url=' . urlencode($rr['xchan_hash']) . '&interactive=0',
+						'follow' => z_root() . '/follow/?f=&url=' . urlencode($rr['xchan_hash']) . '&interactive=1',
 						'connect_hover' => t('Connect at this location'),
-						'role' => $roles_dict[$rr['abook_role']],
+						'role' => $roles_dict[$rr['abook_role']] ?? '',
 						'pending' => intval($rr['abook_pending'])
 					);
 				}
@@ -365,13 +374,13 @@ class Connections extends \Zotlabs\Web\Controller {
 
 		$limit = service_class_fetch(local_channel(),'total_channels');
 		if($limit !== false) {
-			$abook_usage_message = sprintf( t("You have %1$.0f of %2$.0f allowed connections."), $$total, $limit);
+			$abook_usage_message = sprintf( t("You have %1$.0f of %2$.0f allowed connections."), $total, $limit);
 		}
 		else {
 			$abook_usage_message = '';
  		}
 
-		if($_REQUEST['aj']) {
+		if(isset($_REQUEST['aj']) && $_REQUEST['aj']) {
 			if($contacts) {
 				$o = replace_macros(get_markup_template('contactsajax.tpl'),array(
 					'$contacts' => $contacts,
@@ -393,14 +402,13 @@ class Connections extends \Zotlabs\Web\Controller {
 				'$search' => $search_hdr,
 				'$label' => t('Search'),
 				'$role_label' => t('Contact role'),
-				'$desc' => t('Search your connections'),
+				'$desc' => $search ?? t('Search your connections'),
 				'$finding' => (($searching) ? t('Contact search') . ": '" . $search . "'" : ""),
 				'$submit' => t('Find'),
 				'$edit' => t('Edit'),
 				'$approve' => t('Approve'),
 				'$cmd' => App::$cmd,
 				'$contacts' => $contacts,
-				'$paginate' => paginate($a),
 				'$abook_usage_message' => $abook_usage_message,
 				'$group_label' => t('This is a group/forum channel')
 			]);

Zotlabs/Module/Contactedit.php

@@ -103,7 +103,7 @@ class Contactedit extends Controller {
 				dbesc($profile_id),
 				intval(local_channel())
 			);
-			if (!count($r)) {
+			if (!$r) {
 				notice(t('Could not locate selected profile.') . EOL);
 				return;
 			}
@@ -452,8 +452,8 @@ class Contactedit extends Controller {
 
 		if (is_ajax()) {
 			json_return_and_die([
-				'success' => ((intval($_REQUEST['success'])) ? intval($_REQUEST['success']) : 1),
-				'message' => (($_REQUEST['success']) ? t('Contact updated') : t('Contact update failed')),
+				'success' => ((isset($_REQUEST['success'])) ? intval($_REQUEST['success']) : 1),
+				'message' => ((isset($_REQUEST['success'])) ? t('Contact updated') : t('Contact update failed')),
 				'id' => $contact_id,
 				'title' => $header_html,
 				'role' => ((intval($contact['abook_pending'])) ? '' : $roles_dict[$current_permcat]),

Zotlabs/Module/Conversation.php

@@ -0,0 +1,188 @@
+<?php
+
+namespace Zotlabs\Module;
+
+use Zotlabs\Web\Controller;
+use Zotlabs\Lib\ActivityStreams;
+use Zotlabs\Lib\Activity;
+use Zotlabs\Lib\Libzot;
+use Zotlabs\Web\HTTPSig;
+use Zotlabs\Lib\LDSignatures;
+use Zotlabs\Lib\ThreadListener;
+use Zotlabs\Lib\Config;
+use App;
+
+class Conversation extends Controller {
+
+	public function init() {
+
+		if (ActivityStreams::is_as_request()) {
+			$item_id = argv(1);
+
+			if (!$item_id) {
+				http_status_exit(404, 'Not found');
+			}
+
+			$portable_id = EMPTY_STR;
+
+			$item_normal_extra = sprintf(" and not verb in ('%s', '%s') ",
+				dbesc(ACTIVITY_FOLLOW),
+				dbesc(ACTIVITY_UNFOLLOW)
+			);
+
+			$item_normal = " and item.item_hidden = 0 and item.item_type = 0 and item.item_unpublished = 0 and item.item_delayed = 0 and item.item_blocked = 0 $item_normal_extra ";
+
+			$i = null;
+
+			// do we have the item (at all)?
+
+			$r = q("select parent_mid from item where mid = '%s' or uuid = '%s' $item_normal order by item_wall desc limit 1",
+				dbesc(z_root() . '/item/' . $item_id),
+				dbesc($item_id)
+			);
+
+			if (!$r) {
+				http_status_exit(404, 'Not found');
+			}
+
+			// process an authenticated fetch
+
+			$sigdata = HTTPSig::verify(EMPTY_STR);
+			if ($sigdata['portable_id'] && $sigdata['header_valid']) {
+				$portable_id = $sigdata['portable_id'];
+				observer_auth($portable_id);
+
+				// first see if we have a copy of this item's parent owned by the current signer
+				// include xchans for all zot-like networks - these will have the same guid and public key
+
+				$x = q(
+					"select * from xchan where xchan_hash = '%s'",
+					dbesc($sigdata['portable_id'])
+				);
+
+				if ($x) {
+					$xchans = q(
+						"select xchan_hash from xchan where xchan_hash = '%s' OR ( xchan_guid = '%s' AND xchan_pubkey = '%s' ) ",
+						dbesc($sigdata['portable_id']),
+						dbesc($x[0]['xchan_guid']),
+						dbesc($x[0]['xchan_pubkey'])
+					);
+
+					if ($xchans) {
+						$hashes = ids_to_querystr($xchans, 'xchan_hash', true);
+						$i = q(
+							"select id as item_id from item where mid = '%s' $item_normal and owner_xchan in ( " . protect_sprintf($hashes) . " ) limit 1",
+							dbesc($r[0]['parent_mid'])
+						);
+					}
+				}
+			}
+			elseif (Config::get('system', 'require_authenticated_fetch', false)) {
+				http_status_exit(403, 'Permission denied');
+			}
+
+			// if we don't have a parent id belonging to the signer see if we can obtain one as a visitor that we have permission to access
+			// with a bias towards those items owned by channels on this site (item_wall = 1)
+
+			$sql_extra = item_permissions_sql(0);
+
+			if (!$i) {
+				$i = q("select id as item_id from item where mid = '%s' $item_normal $sql_extra order by item_wall desc limit 1",
+					dbesc($r[0]['parent_mid'])
+				);
+			}
+
+			if (!$i) {
+				http_status_exit(403, 'Forbidden');
+			}
+
+			$parents_str = ids_to_querystr($i, 'item_id');
+			$itemspage = 30;
+			$page = $_REQUEST['page'] ?? 1;
+			$offset = (intval($page) - 1) * $itemspage;
+
+			$items = dbq("SELECT item.*,
+				item.id AS item_id,
+				(SELECT count(*) FROM item WHERE item.parent IN ( $parents_str ) $item_normal) AS total
+				FROM item WHERE item.parent IN ( $parents_str ) $item_normal order by item.id LIMIT $itemspage OFFSET $offset"
+			);
+
+			if (!$items) {
+				http_status_exit(404, 'Not found');
+			}
+
+			xchan_query($items, true);
+			$items = fetch_post_tags($items);
+
+			$observer = App::get_observer();
+			$parent = $items[0];
+			$recips = (($parent['owner']['xchan_network'] === 'activitypub') ? get_iconfig($parent['id'], 'activitypub', 'recips', []) : []);
+			$to = (($recips && array_key_exists('to', $recips) && is_array($recips['to'])) ? $recips['to'] : null);
+			$nitems = [];
+			foreach ($items as $i) {
+				$mids = [];
+
+				if (intval($i['item_private'])) {
+					if (!$observer) {
+						continue;
+					}
+					/*
+					// ignore private reshare, possibly from hubzilla
+					if ($i['verb'] === 'Announce') {
+						if (!in_array($i['thr_parent'], $mids)) {
+							$mids[] = $i['thr_parent'];
+						}
+						continue;
+					}
+					// also ignore any children of the private reshares
+					if (in_array($i['thr_parent'], $mids)) {
+						continue;
+					}
+					*/
+
+					if ((!$to) || (!in_array($observer['xchan_url'], $to))) {
+						continue;
+					}
+				}
+				$nitems[] = $i;
+			}
+
+			if (!$nitems) {
+				http_status_exit(404, 'Not found');
+			}
+
+			App::set_pager_total($nitems[0]['total']);
+			App::set_pager_itemspage($itemspage);
+
+			$channel = channelx_by_n($nitems[0]['uid']);
+
+			if (!$channel) {
+				http_status_exit(404, 'Not found');
+			}
+
+			if (!perm_is_allowed($channel['channel_id'], get_observer_hash(), 'view_stream')) {
+				http_status_exit(403, 'Forbidden');
+			}
+
+			$i = Activity::encode_item_collection($nitems, App::$query_string, 'OrderedCollection', $nitems[0]['total']);
+
+			if (!$i) {
+				http_status_exit(404, 'Not found');
+			}
+
+			if ($portable_id && (!intval($nitems[0]['item_private']))) {
+				$c = q("select abook_id from abook where abook_channel = %d and abook_xchan = '%s'",
+					intval($nitems[0]['uid']),
+					dbesc($portable_id)
+				);
+				if (!$c) {
+					ThreadListener::store(z_root() . '/item/' . $item_id, $portable_id);
+				}
+			}
+
+			as_return_and_die($i, $channel);
+		}
+
+		goaway(z_root() . '/item/' . argv(1));
+	}
+}

Zotlabs/Module/Cover_photo.php

@@ -3,7 +3,7 @@ namespace Zotlabs\Module;
 
 use Zotlabs\Lib\Libsync;
 
-/* 
+/*
    @file cover_photo.php
    @brief Module-file with functions for handling of cover-photos
 
@@ -29,43 +29,43 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 		if(! local_channel()) {
 			return;
 		}
-	
+
 		$channel = \App::get_channel();
-		profile_load($channel['channel_address']);	
+		profile_load($channel['channel_address']);
 	}
-	
+
 	/**
 	 * @brief Evaluate posted values
 	 *
 	 * @return void
 	 *
 	 */
-	
+
 	function post() {
-	
+
 		if(! local_channel()) {
 			return;
 		}
-	
+
 		$channel = \App::get_channel();
-		
+
 		check_form_security_token_redirectOnErr('/cover_photo', 'cover_photo');
 
 		// Remove cover photo
 		if(isset($_POST['remove'])) {
-			
+
 			$r = q("SELECT resource_id FROM photo WHERE photo_usage = %d AND uid = %d LIMIT 1",
 				intval(PHOTO_COVER),
 				intval(local_channel())
 			);
-			
+
 			if($r) {
 			    q("update photo set photo_usage = %d where photo_usage = %d and uid = %d",
 			    	intval(PHOTO_NORMAL),
 				    intval(PHOTO_COVER),
 				    intval(local_channel())
 			    );
-			
+
 				$sync = attach_export_data($channel,$r[0]['resource_id']);
 				if($sync)
 				    Libsync::build_sync_packet($channel['channel_id'],array('file' => array($sync)));
@@ -76,47 +76,47 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 
 			goaway(z_root() . '/cover_photo');
 		}
-	        
+
 		if((array_key_exists('cropfinal',$_POST)) && ($_POST['cropfinal'] == 1)) {
-	
+
 			// phase 2 - we have finished cropping
-	
+
 			if(argc() != 2) {
 				notice( t('Image uploaded but image cropping failed.') . EOL );
 				return;
 			}
-	
+
 			$image_id = argv(1);
-	
+
 			if(substr($image_id,-2,1) == '-') {
 				$scale = substr($image_id,-1,1);
 				$image_id = substr($image_id,0,-2);
 			}
-				
+
 
 
 			$srcX = intval($_POST['xstart']);
 			$srcY = intval($_POST['ystart']);
 			$srcW = intval($_POST['xfinal']) - $srcX;
 			$srcH = intval($_POST['yfinal']) - $srcY;
-	
+
 			$r = q("select gender from profile where uid = %d and is_default = 1 limit 1",
 				intval(local_channel())
 			);
 			if($r) {
 				$profile = $r[0];
 			}
-	
+
 			$r = q("SELECT * FROM photo WHERE resource_id = '%s' AND uid = %d AND imgscale = 0 LIMIT 1",
 				dbesc($image_id),
 				intval(local_channel())
 			);
-	
+
 			if($r) {
 
 				$max_thumb = intval(get_config('system','max_thumbnail',1600));
 				$iscaled = false;
-				if(intval($r[0]['height']) > $max_thumb || intval($r[0]['width']) > $max_thumb) { 
+				if(intval($r[0]['height']) > $max_thumb || intval($r[0]['width']) > $max_thumb) {
 					$imagick_path = get_config('system','imagick_convert_path');
 					if($imagick_path && @file_exists($imagick_path) && intval($r[0]['os_storage'])) {
 
@@ -150,63 +150,63 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 
 				$im = photo_factory($base_image['content'], $base_image['mimetype']);
 				if($im->is_valid()) {
-	
-					// We are scaling and cropping the relative pixel locations to the original photo instead of the 
+
+					// We are scaling and cropping the relative pixel locations to the original photo instead of the
 					// scaled photo we operated on.
-	
+
 					// First load the scaled photo to check its size. (Should probably pass this in the post form and save
 					// a query.)
-	
+
 					$g = q("select width, height from photo where resource_id = '%s' and uid = %d and imgscale = 3",
 						dbesc($image_id),
 						intval(local_channel())
 					);
-	
-	
+
+
 					$scaled_width = $g[0]['width'];
 					$scaled_height = $g[0]['height'];
-	
+
 					if((! $scaled_width) || (! $scaled_height)) {
 						logger('potential divide by zero scaling cover photo');
 						return;
 					}
-	
+
 					// unset all other cover photos
-	
+
 					q("update photo set photo_usage = %d where photo_usage = %d and uid = %d",
 						intval(PHOTO_NORMAL),
 						intval(PHOTO_COVER),
 						intval(local_channel())
 					);
-	
+
 					$orig_srcx = ( $base_image['width'] / $scaled_width ) * $srcX;
 					$orig_srcy = ( $base_image['height'] / $scaled_height ) * $srcY;
 	 				$orig_srcw = ( $srcW / $scaled_width ) * $base_image['width'];
 	 				$orig_srch = ( $srcH / $scaled_height ) * $base_image['height'];
-	
+
 					$im->cropImageRect(1200,435,$orig_srcx, $orig_srcy, $orig_srcw, $orig_srch);
-	
+
 					$aid = get_account_id();
-	
-					$p = [ 
-						'aid'          => $aid, 
-						'uid'          => local_channel(), 
+
+					$p = [
+						'aid'          => $aid,
+						'uid'          => local_channel(),
 						'resource_id'  => $base_image['resource_id'],
-						'filename'     => $base_image['filename'], 
+						'filename'     => $base_image['filename'],
 						'album'        => t('Cover Photos'),
 						'os_path'      => $base_image['os_path'],
 						'display_path' => $base_image['display_path'],
 						'photo_usage'  => PHOTO_COVER
 					];
-					
+
 					$r1 = $im->storeThumbnail($p, PHOTO_RES_COVER_1200);
-	
+
 					$im->doScaleImage(850,310);
 					$r2 = $im->storeThumbnail($p, PHOTO_RES_COVER_850);
-	
+
 					$im->doScaleImage(425,160);
 					$r3 = $im->storeThumbnail($p, PHOTO_RES_COVER_425);
-					
+
 					if($r1 === false || $r2 === false || $r3 === false) {
 						// if one failed, delete them all so we can start over.
 						notice( t('Image resize failed.') . EOL );
@@ -214,7 +214,7 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 							dbesc($base_image['resource_id']),
 							local_channel()
 						);
-						
+
 						$x = q("SELECT content FROM photo WHERE resource_id = '%s' AND uid = %d AND os_storage = 1 AND imgscale >= 7",
 							dbesc($base_image['resource_id']),
 							local_channel()
@@ -229,7 +229,7 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 					}
 
 					$this->send_cover_photo_activity($channel,$base_image,$profile);
-					
+
 					$sync = attach_export_data($channel,$base_image['resource_id']);
 					if($sync)
 					    Libsync::build_sync_packet($channel['channel_id'],array('file' => array($sync)));
@@ -240,33 +240,33 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 				else
 					notice( t('Unable to process image') . EOL);
 			}
-	
+
 			goaway(z_root() . '/channel/' . $channel['channel_address']);
-	
+
 		}
-	
-	
+
+
 		$hash = photo_new_resource();
 		$smallest = 0;
-	
+
 		require_once('include/attach.php');
-	
+
 		$res = attach_store(\App::get_channel(), get_observer_hash(), '', array('album' => t('Cover Photos'), 'hash' => $hash, 'nosync' => true));
-	
+
 		logger('attach_store: ' . print_r($res,true));
-	
+
 		if($res && intval($res['data']['is_photo'])) {
 			$i = q("select * from photo where resource_id = '%s' and uid = %d and imgscale = 0",
 				dbesc($hash),
 				intval(local_channel())
 			);
-	
+
 			if(! $i) {
 				notice( t('Image upload failed.') . EOL );
 				return;
 			}
 			$os_storage = false;
-	
+
 			foreach($i as $ii) {
 				$smallest   = intval($ii['imgscale']);
 				$os_storage = intval($ii['os_storage']);
@@ -274,91 +274,91 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 				$filetype   = $ii['mimetype'];
 			}
 		}
-	
+
 		$imagedata = (($os_storage) ? @file_get_contents(dbunescbin($imagedata)) : dbunescbin($imagedata));
 		$ph = photo_factory($imagedata, $filetype);
-	
+
 		if(! $ph->is_valid()) {
 			notice( t('Unable to process image.') . EOL );
 			return;
 		}
-	
+
 		return $this->cover_photo_crop_ui_head($a, $ph, $hash, $smallest);
-		
+
 	}
-	
+
 	function send_cover_photo_activity($channel,$photo,$profile) {
-	
+
 		$arr = array();
 		$arr['item_thread_top'] = 1;
 		$arr['item_origin'] = 1;
 		$arr['item_wall'] = 1;
-	
+
 		if($profile && stripos($profile['gender'],t('female')) !== false)
 			$t = t('%1$s updated her %2$s');
 		elseif($profile && stripos($profile['gender'],t('male')) !== false)
 			$t = t('%1$s updated his %2$s');
 		else
 			$t = t('%1$s updated their %2$s');
-	
+
 		$ptext = '[zrl=' . z_root() . '/photos/' . $channel['channel_address'] . '/image/' . $photo['resource_id'] . ']' . t('cover photo') . '[/zrl]';
-	
-		$ltext = '[zrl=' . z_root() . '/profile/' . $channel['channel_address'] . ']' . '[zmg]' . z_root() . '/photo/' . $photo['resource_id'] . '-8[/zmg][/zrl]'; 
-	
+
+		$ltext = '[zrl=' . z_root() . '/profile/' . $channel['channel_address'] . ']' . '[zmg]' . z_root() . '/photo/' . $photo['resource_id'] . '-8[/zmg][/zrl]';
+
 		$arr['body'] = sprintf($t,$channel['channel_name'],$ptext) . "\n\n" . $ltext;
-	
+
 		$acl = new \Zotlabs\Access\AccessList($channel);
 		$x = $acl->get();
 		$arr['allow_cid'] = $x['allow_cid'];
-	
+
 		$arr['allow_gid'] = $x['allow_gid'];
 		$arr['deny_cid'] = $x['deny_cid'];
 		$arr['deny_gid'] = $x['deny_gid'];
-	
+
 		$arr['uid'] = $channel['channel_id'];
 		$arr['aid'] = $channel['channel_account_id'];
-	
+
 		$arr['owner_xchan'] = $channel['channel_hash'];
 		$arr['author_xchan'] = $channel['channel_hash'];
-	
+
 		post_activity_item($arr);
-	
-	
+
+
 	}
-	
-	
+
+
 	/**
 	 * @brief Generate content of profile-photo view
 	 *
 	 * @return string
 	 *
 	 */
-	
-	
+
+
 	function get() {
-	
+
 		if(! local_channel()) {
 			notice( t('Permission denied.') . EOL );
 			return;
 		}
-	
+
 		$channel = \App::get_channel();
-	
+
 		$newuser = false;
-	
+
 		if(argc() == 2 && argv(1) === 'new')
 			$newuser = true;
-	
+
 		if(argv(1) === 'use') {
 			if (argc() < 3) {
 				notice( t('Permission denied.') . EOL );
 				return;
 			};
-			
+
 	//		check_form_security_token_redirectOnErr('/cover_photo', 'cover_photo');
-	        
+
 			$resource_id = argv(2);
-	
+
 			$r = q("SELECT id, album, imgscale FROM photo WHERE uid = %d AND resource_id = '%s' ORDER BY imgscale ASC",
 				intval(local_channel()),
 				dbesc($resource_id)
@@ -372,22 +372,22 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 				if($rr['imgscale'] == 7)
 					$havescale = true;
 			}
-	
+
 			$r = q("SELECT content, mimetype, resource_id, os_storage FROM photo WHERE id = %d and uid = %d limit 1",
 				intval($r[0]['id']),
 				intval(local_channel())
-	
+
 			);
 			if(! $r) {
 				notice( t('Photo not available.') . EOL );
 				return;
 			}
-	
+
 			if(intval($r[0]['os_storage']))
 				$data = @file_get_contents(dbunescbin($r[0]['content']));
 			else
-				$data = dbunescbin($r[0]['content']); 
-	
+				$data = dbunescbin($r[0]['content']);
+
 			$ph = photo_factory($data, $r[0]['mimetype']);
 			$smallest = 0;
 			if($ph->is_valid()) {
@@ -396,7 +396,7 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 					dbesc($r[0]['resource_id']),
 					intval(local_channel())
 				);
-	
+
 				if($i) {
 					$hash = $i[0]['resource_id'];
 					foreach($i as $ii) {
@@ -404,15 +404,15 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 					}
 	            }
 	        }
-	 
+
 			$this->cover_photo_crop_ui_head($a, $ph, $hash, $smallest);
 		}
-	
-	
+
+
 		if(! x(\App::$data,'imagecrop')) {
-	
+
 			$tpl = get_markup_template('cover_photo.tpl');
-	
+
 			$o .= replace_macros($tpl,array(
 				'$user'                => \App::$channel['channel_address'],
 				'$info'                => t('Your cover photo may be visible to anybody on the internet'),
@@ -422,7 +422,7 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 				'$title'               => t('Change Cover Photo'),
 				'$submit'              => t('Upload'),
 				'$remove'              => t('Remove'),
-				'$profiles'            => $profiles,
+				'$profiles'            => false,
 				'$embedPhotos' => t('Use a photo from your albums'),
 				'$embedPhotosModalTitle' => t('Use a photo from your albums'),
 				'$embedPhotosModalCancel' => t('Cancel'),
@@ -434,13 +434,13 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 				'$modalerrorlink' => t('Error getting photo link'),
 				'$modalerroralbum' => t('Error getting album'),
 				'$form_security_token' => get_form_security_token("cover_photo"),
-					/// @FIXME - yuk  
+					/// @FIXME - yuk
 				'$select' => t('Select existing photo'),
 
 			));
-			
+
 			call_hooks('cover_photo_content_end', $o);
-			
+
 			return $o;
 		}
 		else {
@@ -459,10 +459,8 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 			));
 			return $o;
 		}
-	
-		return; // NOTREACHED
 	}
-	
+
 	/* @brief Generate the UI for photo-cropping
 	 *
 	 * @param $a Current application
@@ -470,30 +468,30 @@ class Cover_photo extends \Zotlabs\Web\Controller {
 	 * @return void
 	 *
 	 */
-	
+
 	function cover_photo_crop_ui_head(&$a, $ph, $hash, $smallest){
-	
+
 		$max_length = get_config('system','max_image_length');
 		if(! $max_length)
 			$max_length = MAX_IMAGE_LENGTH;
 		if($max_length > 0)
 			$ph->scaleImage($max_length);
-	
+
 		$width  = $ph->getWidth();
 		$height = $ph->getHeight();
-	
+
 		if($width < 300 || $height < 300) {
 			$ph->scaleImageUp(240);
 			$width  = $ph->getWidth();
 			$height = $ph->getHeight();
 		}
-	
-	
+
+
 		\App::$data['imagecrop'] = $hash;
 		\App::$data['imagecrop_resolution'] = $smallest;
 		\App::$page['htmlhead'] .= replace_macros(get_markup_template("crophead.tpl"), array());
 		return;
 	}
-	
-	
+
+
 }

Zotlabs/Module/Defperms.php

@@ -258,7 +258,6 @@ class Defperms extends Controller {
 				'$sections'       => $sections,
 				'$autolbl'        => t('The permissions indicated on this page will be applied to all new connections.'),
 				'$autoapprove'    => t('Automatic approval settings'),
-				'$unapproved'     => $unapproved,
 				'$inherited'      => t('inherited'),
 				'$submit'         => t('Submit'),
 				'$me'             => t('My Settings'),

Zotlabs/Module/Dircensor.php

@@ -4,6 +4,7 @@ namespace Zotlabs\Module;
 
 use App;
 use Zotlabs\Web\Controller;
+use Zotlabs\Lib\Libzotdir;
 
 
 class Dircensor extends Controller {
@@ -15,11 +16,12 @@ class Dircensor extends Controller {
 
 		$dirmode = intval(get_config('system','directory_mode'));
 
-		if (! ($dirmode == DIRECTORY_MODE_PRIMARY || $dirmode == DIRECTORY_MODE_STANDALONE)) {
+		if(!in_array($dirmode, [DIRECTORY_MODE_PRIMARY, DIRECTORY_MODE_SECONDARY, DIRECTORY_MODE_STANDALONE])) {
 			return;
 		}
 
 		$xchan = argv(1);
+
 		if(! $xchan) {
 			return;
 		}
@@ -32,19 +34,43 @@ class Dircensor extends Controller {
 			return;
 		}
 
-		$val = (($r[0]['xchan_censored']) ? 0 : 1);
+		$severity = intval($_REQUEST['severity'] ?? 0);
+		$flag = DIRECTORY_FLAG_OK;
 
-		q("update xchan set xchan_censored = $val where xchan_hash = '%s'",
+		if ($severity === 1) {
+			$flag = DIRECTORY_FLAG_UNSAFE;
+		}
+
+		if ($severity === 2) {
+			$flag = DIRECTORY_FLAG_HIDDEN;
+		}
+
+		Libzotdir::update($xchan, $r[0]['xchan_url'], true, $flag);
+
+		q("UPDATE xchan SET xchan_censored = %d WHERE xchan_hash = '%s'",
+			intval($flag),
 			dbesc($xchan)
 		);
 
-		if($val) {
+		q("UPDATE xtag SET xtag_flags = %d WHERE xtag_hash = '%s'",
+			intval($flag),
+			dbesc($xchan)
+		);
+
+		if($flag) {
 			info( t('Entry censored') . EOL);
 		}
 		else {
-			info( t('Entry uncensored') . EOL);
+			info( t('Entry OK') . EOL);
 		}
-		
+
+		if (isset($_REQUEST['aj'])) {
+			json_return_and_die([
+				'success' => 1,
+				'flag' => $flag
+			]);
+		}
+
 		goaway(z_root() . '/directory');
 
 	}

Zotlabs/Module/Directory.php

@@ -83,7 +83,7 @@ class Directory extends Controller {
 		$globaldir = Libzotdir::get_directory_setting($observer, 'globaldir');
 
 		// override your personal global search pref if we're doing a navbar search of the directory
-		if(intval($_REQUEST['navsearch']))
+		if(isset($_REQUEST['navsearch']) && intval($_REQUEST['navsearch']))
 			$globaldir = 1;
 
 		$safe_mode = Libzotdir::get_directory_setting($observer, 'safemode');
@@ -98,15 +98,18 @@ class Directory extends Controller {
 		else
 			$search = ((x($_GET,'search')) ? notags(trim(rawurldecode($_GET['search']))) : '');
 
-
+		$advanced = '';
 		if(strpos($search,'=') && local_channel() && feature_enabled(local_channel(), 'advanced_dirsearch'))
 			$advanced = $search;
 
-		$keywords = (($_GET['keywords']) ? $_GET['keywords'] : '');
+		$keywords = $_GET['keywords'] ?? '';
 
 		// Suggest channels if no search terms or keywords are given
 		$suggest = (local_channel() && x($_REQUEST,'suggest')) ? $_REQUEST['suggest'] : '';
 
+		$addresses = [];
+		$common = [];
+
 		if($suggest) {
 
 			// the directory options have no effect in suggestion mode
@@ -123,8 +126,7 @@ class Directory extends Controller {
 			}
 
 			// Remember in which order the suggestions were
-			$addresses = array();
-			$common = array();
+
 			$index = 0;
 			foreach($r as $rr) {
 				$common[$rr['xchan_addr']] = ((intval($rr['total']) > 0) ? intval($rr['total']) - 1 : 0);
@@ -132,7 +134,7 @@ class Directory extends Controller {
 			}
 
 			// Build query to get info about suggested people
-			$advanced = '';
+
 			foreach(array_keys($addresses) as $address) {
 				$advanced .= "address=\"$address\" ";
 			}
@@ -147,7 +149,9 @@ class Directory extends Controller {
 
 		$directory_admin = false;
 
-		if(($dirmode == DIRECTORY_MODE_PRIMARY) || ($dirmode == DIRECTORY_MODE_STANDALONE)) {
+		$url = '';
+
+		if(in_array($dirmode, [DIRECTORY_MODE_PRIMARY, DIRECTORY_MODE_SECONDARY, DIRECTORY_MODE_STANDALONE])) {
 			$url = z_root() . '/dirsearch';
 			if (is_site_admin()) {
 				$directory_admin = true;
@@ -187,7 +191,10 @@ class Directory extends Controller {
 			if(get_config('system','disable_directory_keywords'))
 				$kw = 0;
 
-			$query = $url . '?f=&kw=' . $kw . (($safe_mode != 1) ? '&safe=' . $safe_mode : '');
+			if (intval($safe_mode) === 0 && $directory_admin)
+				$safe_mode = -1;
+
+			$query = $url . '?f=&kw=' . $kw . (($safe_mode < 1) ? '&safe=' . $safe_mode : '');
 
 			if($token)
 				$query .= '&t=' . $token;
@@ -228,7 +235,7 @@ class Directory extends Controller {
 				$j = json_decode($x['body'],true);
 				if($j) {
 
-					if($j['results']) {
+					if(isset($j['results']) && $j['results']) {
 
 						$results = $j['results'];
 						if($suggest) {
@@ -254,40 +261,33 @@ class Directory extends Controller {
 								$connect_link = '';
 
 							$location = '';
-							if(strlen($rr['locale']))
+							if(isset($rr['locale']))
 								$location .= $rr['locale'];
-							if(strlen($rr['region'])) {
-								if(strlen($rr['locale']))
+							if(isset($rr['region'])) {
+								if($location)
 									$location .= ', ';
 								$location .= $rr['region'];
 							}
-							if(strlen($rr['country'])) {
-								if(strlen($location))
+							if(isset($rr['country'])) {
+								if($location)
 									$location .= ', ';
 								$location .= $rr['country'];
 							}
 
 							$age = '';
-							if(strlen($rr['birthday'])) {
+							if(isset($rr['birthday'])) {
 								if(($years = age($rr['birthday'],'UTC','')) > 0)
 									$age = $years;
 							}
 
 							$page_type = '';
 
-							$rating_enabled = get_config('system','rating_enabled');
-
-							if($rr['total_ratings'] && $rating_enabled)
-								$total_ratings = sprintf( tt("%d rating", "%d ratings", $rr['total_ratings']), $rr['total_ratings']);
-							else
-								$total_ratings = '';
-
 							$profile = $rr;
 
-							if ((x($profile,'locale') == 1)
-								|| (x($profile,'region') == 1)
-								|| (x($profile,'postcode') == 1)
-								|| (x($profile,'country') == 1))
+					//		if ((x($profile,'locale') == 1)
+					//			|| (x($profile,'region') == 1)
+					//			|| (x($profile,'postcode') == 1)
+					//			|| (x($profile,'country') == 1))
 
 							$gender = ((x($profile,'gender') == 1) ? t('Gender: ') . $profile['gender']: False);
 
@@ -299,10 +299,11 @@ class Directory extends Controller {
 							$hometown = ((x($profile,'hometown') == 1) ? html2plain($profile['hometown'])  : False);
 
 							$about = ((x($profile,'about') == 1) ? zidify_links(bbcode($profile['about'], ['tryoembed' => false])) : False);
-							if ($about && $safe_mode) {
-								$about = html2plain($about);
+							if ($about && $safe_mode > 0) {
+								$about = strip_tags($about, '<br>');
 							}
 
+
 							$keywords = ((x($profile,'keywords')) ? $profile['keywords'] : '');
 
 
@@ -312,7 +313,7 @@ class Directory extends Controller {
 								$keywords = str_replace(',',' ', $keywords);
 								$keywords = str_replace('  ',' ', $keywords);
 								$karr = explode(' ', $keywords);
-
+								$marr = [];
 								if($karr) {
 									if(local_channel()) {
 										$r = q("select keywords from profile where uid = %d and is_default = 1 limit 1",
@@ -352,16 +353,17 @@ class Directory extends Controller {
 								'location' => $location,
 								'location_label' => t('Location:'),
 								'gender'   => $gender,
-								'total_ratings' => $total_ratings,
-								'viewrate' => true,
-								'canrate' => (($rating_enabled && local_channel()) ? true : false),
 								'pdesc'	=> $pdesc,
 								'pdesc_label' => t('Description:'),
-								'censor' => (($directory_admin) ? 'dircensor/' . $rr['hash'] : ''),
-								'censor_label' => (($rr['censored']) ? t('Uncensor') : t('Censor')),
+								'censor' => (($directory_admin) ? 'dircensor/' . $rr['hash'] . '?severity=' . ((intval($rr['censored']) > 0) ? 0 : 1) : ''),
+								'censor_label' => t('Unsafe'),
+								'censor_class' => ((intval($rr['censored']) === 1) ? 'active' : ''),
+								'censor_2' => (($directory_admin) ? 'dircensor/' . $rr['hash'] . '?severity=' . ((intval($rr['censored']) > 1) ? 0 : 2) : ''),
+								'censor_2_label' => t('Spam'),
+								'censor_2_class' => ((intval($rr['censored']) > 1) ? 'active' : ''),
 								'marital'  => $marital,
 								'homepage' => $homepage,
-								'homepageurl' => (($safe_mode) ? $homepageurl : linkify($homepageurl)),
+								'homepageurl' => (($safe_mode  > 0) ? $homepageurl : linkify($homepageurl)),
 								'hometown' => $hometown,
 								'hometown_label' => t('Hometown:'),
 								'about' => $about,
@@ -374,9 +376,9 @@ class Directory extends Controller {
 								'keywords' => $out,
 								'ignlink' => $suggest ? z_root() . '/directory?ignore=' . $rr['hash'] : '',
 								'ignore_label' => t('Don\'t suggest'),
-								'common_friends' => (($common[$rr['address']]) ? intval($common[$rr['address']]) : ''),
+								'common_friends' => $common[$rr['address']] ?? '',
 								'common_label' => t('Common connections (estimated):'),
-								'common_count' => intval($common[$rr['address']]),
+								'common_count' => $common[$rr['address']] ?? '',
 								'safe' => $safe_mode
 							);
 
@@ -402,14 +404,15 @@ class Directory extends Controller {
 
 						ksort($entries); // Sort array by key so that foreach-constructs work as expected
 
-						if($j['keywords']) {
+						if(isset($j['keywords']) && $j['keywords']) {
 							App::$data['directory_keywords'] = $j['keywords'];
 						}
 
 						logger('mod_directory: entries: ' . print_r($entries,true), LOGGER_DATA);
 
+						$aj = $_REQUEST['aj'] ?? '';
 
-						if($_REQUEST['aj']) {
+						if($aj) {
 							if($entries) {
 								$o = replace_macros(get_markup_template('directajax.tpl'),array(
 									'$entries' => $entries
@@ -441,7 +444,8 @@ class Directory extends Controller {
 								'$reverse' => t('Reverse Alphabetic'),
 								'$date' => t('Newest to Oldest'),
 								'$reversedate' => t('Oldest to Newest'),
-								'$suggest' => $suggest ? '&suggest=1' : ''
+								'$suggest' => $suggest ? '&suggest=1' : '',
+								'$directory_admin' => $directory_admin
 							));
 
 
@@ -449,12 +453,12 @@ class Directory extends Controller {
 
 					}
 					else {
-						if($_REQUEST['aj']) {
+						if(isset($_REQUEST['aj']) && $_REQUEST['aj']) {
 							$o = '<div id="content-complete"></div>';
 							echo $o;
 							killme();
 						}
-						if(App::$pager['page'] == 1 && $j['records'] == 0 && strpos($search,'@')) {
+						if(App::$pager['page'] == 1 && (isset($j['records']) && $j['records'] == 0) && strpos($search,'@')) {
 							goaway(z_root() . '/chanview/?f=&address=' . $search);
 						}
 						info( t("No entries (some entries may be hidden).") . EOL);

Zotlabs/Module/Dirsearch.php

@@ -15,8 +15,7 @@ class Dirsearch extends Controller {
 
 		$ret = array('success' => false);
 
-	//	logger('request: ' . print_r($_REQUEST,true));
-
+		// logger('request: ' . print_r($_REQUEST,true));
 
 		$dirmode = intval(get_config('system','directory_mode'));
 
@@ -25,8 +24,7 @@ class Dirsearch extends Controller {
 			json_return_and_die($ret);
 		}
 
-
-		$access_token = $_REQUEST['t'];
+		$access_token = $_REQUEST['t'] ?? '';
 
 		$token = get_config('system','realm_token');
 		if($token && $access_token != $token) {
@@ -41,11 +39,12 @@ class Dirsearch extends Controller {
 		}
 
 		$sql_extra = '';
-
+		$keywords_query = '';
+		$hub_query = '';
 
 		$tables = array('name','address','locale','region','postcode','country','gender','marital','sexual','keywords');
 
-		if($_REQUEST['query']) {
+		if(isset($_REQUEST['query']) && $_REQUEST['query']) {
 			$advanced = $this->dir_parse_query($_REQUEST['query']);
 			if($advanced) {
 				foreach($advanced as $adv) {
@@ -61,22 +60,23 @@ class Dirsearch extends Controller {
 			}
 		}
 
-		$hash     = ((x($_REQUEST['hash']))    ? $_REQUEST['hash']     : '');
+		$hash     = $_REQUEST['hash'] ?? '';
+
+		$name     = $_REQUEST['name'] ?? '';
+		$hub      = $_REQUEST['hub'] ?? '';
+		$address  = $_REQUEST['address'] ?? '';
+		$locale   = $_REQUEST['locale'] ?? '';
+		$region   = $_REQUEST['region'] ?? '';
+		$postcode = $_REQUEST['postcode'] ?? '';
+		$country  = $_REQUEST['country'] ?? '';
+		$gender   = $_REQUEST['gender'] ?? '';
+		$marital  = $_REQUEST['marital'] ?? '';
+		$sexual   = $_REQUEST['sexual'] ?? '';
+		$keywords = $_REQUEST['keywords'] ?? '';
+		$agege    = $_REQUEST['agege'] ?? 0;
+		$agele    = $_REQUEST['agele'] ?? 0;
+		$kw       = $_REQUEST['kw'] ?? 0;
 
-		$name     = ((x($_REQUEST,'name'))     ? $_REQUEST['name']     : '');
-		$hub      = ((x($_REQUEST,'hub'))      ? $_REQUEST['hub']      : '');
-		$address  = ((x($_REQUEST,'address'))  ? $_REQUEST['address']  : '');
-		$locale   = ((x($_REQUEST,'locale'))   ? $_REQUEST['locale']   : '');
-		$region   = ((x($_REQUEST,'region'))   ? $_REQUEST['region']   : '');
-		$postcode = ((x($_REQUEST,'postcode')) ? $_REQUEST['postcode'] : '');
-		$country  = ((x($_REQUEST,'country'))  ? $_REQUEST['country']  : '');
-		$gender   = ((x($_REQUEST,'gender'))   ? $_REQUEST['gender']   : '');
-		$marital  = ((x($_REQUEST,'marital'))  ? $_REQUEST['marital']  : '');
-		$sexual   = ((x($_REQUEST,'sexual'))   ? $_REQUEST['sexual']   : '');
-		$keywords = ((x($_REQUEST,'keywords')) ? $_REQUEST['keywords'] : '');
-		$agege    = ((x($_REQUEST,'agege'))    ? intval($_REQUEST['agege']) : 0 );
-		$agele    = ((x($_REQUEST,'agele'))    ? intval($_REQUEST['agele']) : 0 );
-		$kw       = ((x($_REQUEST,'kw'))       ? intval($_REQUEST['kw'])    : 0 );
 		$forums   = ((array_key_exists('pubforums',$_REQUEST)) ? intval($_REQUEST['pubforums']) : 0);
 
 		if(get_config('system','disable_directory_keywords'))
@@ -84,9 +84,7 @@ class Dirsearch extends Controller {
 
 
 		// by default use a safe search
-		$safe     = ((x($_REQUEST,'safe')));    // ? intval($_REQUEST['safe'])  : 1 );
-		if ($safe === false)
-				$safe = 1;
+		$safe = $_REQUEST['safe'] ?? 1;
 
 		if(array_key_exists('sync',$_REQUEST)) {
 			if($_REQUEST['sync'])
@@ -101,15 +99,15 @@ class Dirsearch extends Controller {
 			$hub = \App::get_hostname();
 		}
 
-		if($hub)
-			$hub_query = " and xchan_hash in (select hubloc_hash from hubloc where hubloc_host =  '" . protect_sprintf(dbesc($hub)) . "') ";
-		else
-			$hub_query = '';
+		if($hub) {
+			$hub_query = " and xchan_hash in (select hubloc_hash from hubloc where hubloc_deleted = 0 and hubloc_host =  '" . protect_sprintf(dbesc($hub)) . "') ";
+		}
+
 
 		$sort_order  = ((x($_REQUEST,'order')) ? $_REQUEST['order'] : '');
 
 		$joiner = ' OR ';
-		if($_REQUEST['and'])
+		if(isset($_REQUEST['and']) && $_REQUEST['and'])
 			$joiner = ' AND ';
 
 		if($name)
@@ -130,9 +128,16 @@ class Dirsearch extends Controller {
 			$sql_extra .= $this->dir_query_build($joiner,'xprof_marital',$marital);
 		if($sexual)
 			$sql_extra .= $this->dir_query_build($joiner,'xprof_sexual',$sexual);
-		if($keywords)
+		if($keywords && $name) {
+			// this is a general search
 			$sql_extra .= $this->dir_query_build($joiner,'xprof_keywords',$keywords);
-
+		}
+		if($keywords && !$name) {
+			// this is a search for keywords only
+			$keywords_arr = explode(',', $keywords);
+			stringify_array_elms($keywords_arr, true);
+			$keywords_query = " AND xchan_hash IN (SELECT xtag_hash FROM xtag WHERE xtag_term IN (" . protect_sprintf(implode(',', $keywords_arr)) . ")) ";
+		}
 
 		// we only support an age range currently. You must set both agege
 		// (greater than or equal) and agele (less than or equal)
@@ -148,11 +153,11 @@ class Dirsearch extends Controller {
 		}
 
 
-	    $perpage      = (($_REQUEST['n'])              ? $_REQUEST['n']                    : 60);
-	    $page         = (($_REQUEST['p'])              ? intval($_REQUEST['p'] - 1)        : 0);
+	    $perpage      = $_REQUEST['n'] ?? 60;
+	    $page         = ((isset($_REQUEST['p']) && $_REQUEST['p']) ? intval($_REQUEST['p'] - 1) : 0);
 	    $startrec     = (($page+1) * $perpage) - $perpage;
-		$limit        = (($_REQUEST['limit'])          ? intval($_REQUEST['limit'])        : 0);
-		$return_total = ((x($_REQUEST,'return_total')) ? intval($_REQUEST['return_total']) : 0);
+		$limit        = $_REQUEST['limit'] ?? 0;
+		$return_total = $_REQUEST['return_total'] ?? 0;
 
 		// mtime is not currently working
 
@@ -176,9 +181,15 @@ class Dirsearch extends Controller {
 			$sql_extra .= " and xchan_addr like '%%" . \App::get_hostname() . "' ";
 		}
 
-		$safesql = (($safe > 0) ? " and xchan_censored = 0 and xchan_selfcensored = 0 " : '');
+		$safesql = '';
+		if($safe > 0)
+			$safesql = " and xchan_censored = 0 and xchan_selfcensored = 0 ";
+
+		if($safe < 1)
+			$safesql = " and xchan_censored < 2 and xchan_selfcensored < 2 ";
+
 		if($safe < 0)
-			$safesql = " and ( xchan_censored = 1 OR xchan_selfcensored = 1 ) ";
+			$safesql = " and xchan_censored < 3 and xchan_selfcensored < 2 ";
 
 		if($forums)
 			$safesql .= " and xchan_pubforum = " . ((intval($forums)) ? '1 ' : '0 ');
@@ -214,26 +225,25 @@ class Dirsearch extends Controller {
 
 		if($sync) {
 			$spkt = array('transactions' => array());
-			$r = q("select * from updates where ud_date >= '%s' and ud_guid != '' and ud_addr != '' order by ud_date desc",
+
+			$r = q("SELECT * FROM updates WHERE ud_update = 0 AND ud_last = '%s' AND ud_date >= '%s' ORDER BY ud_date DESC",
+				dbesc(NULL_DATE),
 				dbesc($sync)
 			);
+
 			if($r) {
 				foreach($r as $rr) {
-					$flags = array();
-					if($rr['ud_flags'] & UPDATE_FLAGS_DELETED)
-						$flags[] = 'deleted';
-					if($rr['ud_flags'] & UPDATE_FLAGS_FORCED)
-						$flags[] = 'forced';
-
-					$spkt['transactions'][] = array(
+					$spkt['transactions'][] = [
 						'hash' => $rr['ud_hash'],
 						'address' => $rr['ud_addr'],
-						'transaction_id' => $rr['ud_guid'],
+						'host' => $rr['ud_host'],
+						'transaction_id' => $rr['ud_host'], // deprecated 2023-04-12 - can be removed after dir servers at version >= 8.4
 						'timestamp' => $rr['ud_date'],
-						'flags' => $flags
-					);
+						'flags' => $rr['ud_flags']
+					];
 				}
 			}
+
 			json_return_and_die($spkt);
 		}
 		else {
@@ -262,8 +272,8 @@ class Dirsearch extends Controller {
 				xprof.xprof_homepage as homepage,
 				xprof.xprof_hometown as hometown,
 				xprof.xprof_keywords as keywords
-				from xchan left join xprof on xchan_hash = xprof_hash left join hubloc on hubloc_hash = xchan_hash
-				where hubloc_primary = 1 and hubloc_updated > %s - INTERVAL %s and ( $logic $sql_extra ) $hub_query and xchan_network = 'zot6' and xchan_system = 0 and xchan_hidden = 0 and xchan_orphan = 0 and xchan_deleted = 0
+				from xchan left join xprof on xchan_hash = xprof_hash left join hubloc on (hubloc_id_url = xchan_url and hubloc_hash = xchan_hash)
+				where hubloc_primary = 1 and hubloc_updated > %s - INTERVAL %s and ( $logic $sql_extra ) $hub_query $keywords_query and xchan_network = 'zot6' and xchan_system = 0 and xchan_hidden = 0 and xchan_orphan = 0 and xchan_deleted = 0 and hubloc_deleted = 0
 				$safesql $order $qlimit",
 				db_utcnow(),
 				db_quoteinterval('30 DAY')

Zotlabs/Module/Display.php

@@ -2,15 +2,13 @@
 namespace Zotlabs\Module;
 
 use App;
+use Zotlabs\Web\Controller;
 
-require_once("include/bbcode.php");
-require_once('include/security.php');
-require_once('include/conversation.php');
-require_once('include/acl_selectors.php');
-require_once('include/items.php');
+class Display extends Controller {
 
+	function init() {
 
-class Display extends \Zotlabs\Web\Controller {
+	}
 
 	function get($update = 0, $load = false) {
 
@@ -36,7 +34,7 @@ class Display extends \Zotlabs\Web\Controller {
 			}
 		}
 
-		if($_REQUEST['mid']) {
+		if(isset($_REQUEST['mid']) && $_REQUEST['mid']) {
 			$item_hash = $_REQUEST['mid'];
 		}
 
@@ -56,6 +54,8 @@ class Display extends \Zotlabs\Web\Controller {
 
 		$observer_is_owner = false;
 
+		$o = '';
+
 		if(local_channel() && (! $update)) {
 
 			$channel = App::get_channel();
@@ -72,7 +72,7 @@ class Display extends \Zotlabs\Web\Controller {
 				'allow_location'      => ((intval(get_pconfig($channel['channel_id'],'system','use_browser_location'))) ? '1' : ''),
 				'default_location'    => $channel['channel_location'],
 				'nickname'            => $channel['channel_address'],
-				'lockstate'           => (($group || $cid || $channel['channel_allow_cid'] || $channel['channel_allow_gid'] || $channel['channel_deny_cid'] || $channel['channel_deny_gid']) ? 'lock' : 'unlock'),
+				'lockstate'           => (($channel['channel_allow_cid'] || $channel['channel_allow_gid'] || $channel['channel_deny_cid'] || $channel['channel_deny_gid']) ? 'lock' : 'unlock'),
 				'acl'                 => populate_acl($channel_acl,true, \Zotlabs\Lib\PermissionDescription::fromGlobalPermission('view_stream'), get_post_aclDialogDescription(), 'acl_dialog_post'),
 				'permissions'         => $channel_acl,
 				'bang'                => '',
@@ -87,8 +87,9 @@ class Display extends \Zotlabs\Web\Controller {
 				'reset'               => t('Reset form')
 			);
 
-			$o = '<div id="jot-popup">';
-			$o .= status_editor($a,$x,false,'Display');
+			$o .= '<div id="jot-popup">';
+			$a = '';
+			$o .= status_editor($x, false, 'Display');
 			$o .= '</div>';
 		}
 
@@ -107,17 +108,21 @@ class Display extends \Zotlabs\Web\Controller {
 			dbesc($item_hash)
 		);
 
-		if($r) {
-			$target_item = $r[0];
+		if (!$r) {
+			notice( t('Item not found.') . EOL);
+			return '';
 		}
 
+		$target_item = $r[0];
+
+		/* not yet ready for prime time
 		$x = q("select * from xchan where xchan_hash = '%s' limit 1",
 			dbesc($target_item['author_xchan'])
 		);
 		if($x) {
-// not yet ready for prime time
-//			App::$poi = $x[0];
+			App::$poi = $x[0];
 		}
+		*/
 
 		//if the item is to be moderated redirect to /moderate
 		if($target_item['item_blocked'] == ITEM_MODERATED) {
@@ -143,48 +148,8 @@ class Display extends \Zotlabs\Web\Controller {
 			 	return '';
 			}
 		}
-		if($target_item['item_type']  == ITEM_TYPE_ARTICLE) {
-			$x = q("select * from channel where channel_id = %d limit 1",
-				intval($target_item['uid'])
-			);
-			$y = q("select * from iconfig left join item on iconfig.iid = item.id
-				where item.uid = %d and iconfig.cat = 'system' and iconfig.k = 'ARTICLE' and item.id = %d limit 1",
-				intval($target_item['uid']),
-				intval($target_item['parent'])
-			);
-			if($x && $y) {
-				goaway(z_root() . '/articles/' . $x[0]['channel_address'] . '/' . $y[0]['v']);
-			}
-			else {
-				notice( t('Page not found.') . EOL);
-			 	return '';
-			}
-		}
-		if($target_item['item_type']  == ITEM_TYPE_CARD) {
 
-			$x = q("select * from channel where channel_id = %d limit 1",
-				intval($target_item['uid'])
-			);
-
-			$y = q("select * from iconfig left join item on iconfig.iid = item.id
-				where item.uid = %d and iconfig.cat = 'system' and iconfig.k = 'CARD' and item.id = %d limit 1",
-				intval($target_item['uid']),
-				intval($target_item['parent'])
-			);
-
-			if($x && $y) {
-				goaway(z_root() . '/cards/' . $x[0]['channel_address'] . '/' . $y[0]['v']);
-			}
-			else {
-				notice( t('Page not found.') . EOL);
-			 	return '';
-			}
-		}
-		if ($target_item['item_type'] == ITEM_TYPE_CUSTOM) {
-			call_hooks('item_custom_display',$target_item);
-			notice( t('Page not found.') . EOL);
-			return '';
-		}
+		call_hooks('item_custom_display', $target_item);
 
 		$simple_update = '';
 		if($update && $_SESSION['loadtime'])
@@ -246,27 +211,27 @@ class Display extends \Zotlabs\Web\Controller {
 		$observer_hash = get_observer_hash();
 		$item_normal = item_normal();
 		$item_normal_update = item_normal_update();
-
-		$sql_extra = ((local_channel()) ? EMPTY_STR : item_permissions_sql(0, $observer_hash));
+		$sql_extra = '';
+		$r = [];
 
 		if($noscript_content || $load) {
-
 			require_once('include/channel.php');
 			$sys = get_sys_channel();
 			// in case somebody turned off public access to sys channel content using permissions
 			// make that content unsearchable by ensuring the owner uid can't match
 			$sys_id = perm_is_allowed($sys['channel_id'], $observer_hash, 'view_stream') ? $sys['channel_id'] : 0;
 
-			$r = null;
-
 			if(local_channel()) {
 				$r = q("SELECT item.id AS item_id FROM item WHERE uid = %d AND mid = '%s' $item_normal LIMIT 1",
 					intval(local_channel()),
 					dbesc($target_item['parent_mid'])
 				);
+
 			}
 
 			if(!$r) {
+				$sql_extra = item_permissions_sql(0, $observer_hash);
+
 				$r = q("SELECT item.id AS item_id FROM item
 					WHERE ((mid = '%s'
 					AND (((( item.allow_cid = '' AND item.allow_gid = '' AND item.deny_cid  = ''
@@ -290,7 +255,6 @@ class Display extends \Zotlabs\Web\Controller {
 			// make that content unsearchable by ensuring the owner uid can't match
 			$sys_id = perm_is_allowed($sys['channel_id'], $observer_hash, 'view_stream') ? $sys['channel_id'] : 0;
 
-			$r = null;
 			if(local_channel()) {
 				$r = q("SELECT item.parent AS item_id from item
 					WHERE uid = %d
@@ -303,7 +267,9 @@ class Display extends \Zotlabs\Web\Controller {
 				);
 			}
 
-			if(! $r) {
+			if(!$r) {
+				$sql_extra = item_permissions_sql(0, $observer_hash);
+
 				$r = q("SELECT item.id as item_id from item
 					WHERE ((parent_mid = '%s'
 					AND (((( item.allow_cid = '' AND item.allow_gid = '' AND item.deny_cid  = ''
@@ -320,10 +286,6 @@ class Display extends \Zotlabs\Web\Controller {
 			}
 		}
 
-		else {
-			$r = [];
-		}
-
 		if($r) {
 			$parents_str = ids_to_querystr($r,'item_id');
 			if($parents_str) {
@@ -341,71 +303,71 @@ class Display extends \Zotlabs\Web\Controller {
 			$items = array();
 		}
 
-
 		switch($module_format) {
 
-		case 'html':
+			case 'html':
 
-			if ($update) {
-				$o .= conversation($items, 'display', $update, 'client');
-			}
-			else {
-				$o .= '<noscript>';
-				if($noscript_content) {
-					$o .= conversation($items, 'display', $update, 'traditional');
+				if ($update) {
+					$o .= conversation($items, 'display', $update, 'client');
 				}
 				else {
-					$o .= '<div class="section-content-warning-wrapper">' . t('You must enable javascript for your browser to be able to view this content.') . '</div>';
+					$o .= '<noscript>';
+					if($noscript_content) {
+						$o .= conversation($items, 'display', $update, 'traditional');
+					}
+					else {
+						$o .= '<div class="section-content-warning-wrapper">' . t('You must enable javascript for your browser to be able to view this content.') . '</div>';
+					}
+					$o .= '</noscript>';
+
+					if (isset($items[0]['title'])) {
+						App::$page['title'] = $items[0]['title'] . ' - ' . App::$page['title'];
+					}
+
+					$o .= conversation($items, 'display', $update, 'client');
 				}
-				$o .= '</noscript>';
 
-				App::$page['title'] = (($items[0]['title']) ? $items[0]['title'] . " - " . App::$page['title'] : App::$page['title']);
+				break;
 
-				$o .= conversation($items, 'display', $update, 'client');
-			}
+			case 'atom':
 
-			break;
+				$atom = replace_macros(get_markup_template('atom_feed.tpl'), array(
+					'$version'       => xmlify(\Zotlabs\Lib\System::get_project_version()),
+					'$generator'     => xmlify(\Zotlabs\Lib\System::get_platform_name()),
+					'$generator_uri' => 'https://hubzilla.org',
+					'$feed_id'       => xmlify(App::$cmd),
+					'$feed_title'    => xmlify(t('Article')),
+					'$feed_updated'  => xmlify(datetime_convert('UTC', 'UTC', 'now', ATOM_TIME)),
+					'$author'        => '',
+					'$owner'         => '',
+					'$profile_page'  => xmlify(z_root() . '/display/' . gen_link_id($target_item['mid'])),
+				));
 
-		case 'atom':
+				$x = [ 'xml' => $atom, 'channel' => $channel, 'observer_hash' => $observer_hash, 'params' => $params ];
+				call_hooks('atom_feed_top',$x);
 
-			$atom = replace_macros(get_markup_template('atom_feed.tpl'), array(
-				'$version'       => xmlify(\Zotlabs\Lib\System::get_project_version()),
-				'$generator'     => xmlify(\Zotlabs\Lib\System::get_platform_name()),
-				'$generator_uri' => 'https://hubzilla.org',
-				'$feed_id'       => xmlify(App::$cmd),
-				'$feed_title'    => xmlify(t('Article')),
-				'$feed_updated'  => xmlify(datetime_convert('UTC', 'UTC', 'now', ATOM_TIME)),
-				'$author'        => '',
-				'$owner'         => '',
-				'$profile_page'  => xmlify(z_root() . '/display/' . gen_link_id($target_item['mid'])),
-			));
+				$atom = $x['xml'];
 
-			$x = [ 'xml' => $atom, 'channel' => $channel, 'observer_hash' => $observer_hash, 'params' => $params ];
-			call_hooks('atom_feed_top',$x);
-
-			$atom = $x['xml'];
-
-			// a much simpler interface
-			call_hooks('atom_feed', $atom);
+				// a much simpler interface
+				call_hooks('atom_feed', $atom);
 
 
-			if($items) {
-				$type = 'html';
-				foreach($items as $item) {
-					if($item['item_private'])
-						continue;
-					$atom .= atom_entry($item, $type, null, '', true, '', false);
+				if($items) {
+					$type = 'html';
+					foreach($items as $item) {
+						if($item['item_private'])
+							continue;
+						$atom .= atom_entry($item, $type, null, '', true, '', false);
+					}
 				}
-			}
 
-			call_hooks('atom_feed_end', $atom);
+				call_hooks('atom_feed_end', $atom);
 
-			$atom .= '</feed>' . "\r\n";
-
-			header('Content-type: application/atom+xml');
-			echo $atom;
-			killme();
+				$atom .= '</feed>' . "\r\n";
 
+				header('Content-type: application/atom+xml');
+				echo $atom;
+				killme();
 		}
 
 		$o .= '<div id="content-complete"></div>';

Zotlabs/Module/Dreport.php

@@ -56,10 +56,12 @@ class Dreport extends \Zotlabs\Web\Controller {
 			return;
 		}
 
-		$r = q("select * from dreport where dreport_xchan = '%s' and (dreport_mid = '%s' or dreport_mid = '%s')",
+		$r = q("select * from dreport where dreport_xchan = '%s' and (dreport_mid = '%s' or dreport_mid = '%s' or dreport_mid = '%s' or dreport_mid = '%s')",
 			dbesc($channel['channel_hash']),
 			dbesc($mid),
-			dbesc(str_replace('/item/', '/activity/', $mid))
+			dbesc($mid . '#sync'),
+			dbesc(str_replace('/item/', '/activity/', $mid)),
+			dbesc(str_replace('/item/', '/activity/', $mid) . '#sync')
 		);
 
 		if(! $r) {
@@ -94,6 +96,7 @@ class Dreport extends \Zotlabs\Web\Controller {
 				case 'updated':
 					$r[$x]['gravity'] = 5;
 					$r[$x]['dreport_result'] = t('updated');
+					break;
 				case 'update ignored':
 					$r[$x]['gravity'] = 6;
 					$r[$x]['dreport_result'] = t('update ignored');

Zotlabs/Module/Editblock.php

@@ -132,7 +132,7 @@ class Editblock extends \Zotlabs\Web\Controller {
 			'bbcode' => (($mimetype  == 'text/bbcode') ? true : false)
 		);
 
-		$editor = status_editor($a, $x, false, 'Editblock');
+		$editor = status_editor($x, false, 'Editblock');
 
 		$o .= replace_macros(get_markup_template('edpost_head.tpl'), array(
 			'$title' => t('Edit Block'),

Zotlabs/Module/Editlayout.php

@@ -131,7 +131,7 @@ class Editlayout extends \Zotlabs\Web\Controller {
 			'profile_uid' => intval($owner),
 		);
 
-		$editor = status_editor($a, $x, false, 'Editlayout');
+		$editor = status_editor($x, false, 'Editlayout');
 
 		$o .= replace_macros(get_markup_template('edpost_head.tpl'), array(
 			'$title' => t('Edit Layout'),