-----------------------------------------------------------------------------------
- epgd
-
- This daemon is used to download EPG data from the internet and manage it in a mysql database.
-
- Written by: C++/SQL         - JÃ¶rg Wendel (vdr at jwendel dot de)
-             SQL/Procedures  - Christian Kaiser
-             Documetation    - Ingo Prochaska
-
- Homepage:    http://projects.vdr-developer.org/projects/vdr-epg-daemon
- Source-code: http://projects.vdr-developer.org/git/vdr-epg-daemon.git
-
- See the file COPYING for license information.
-----------------------------------------------------------------------------------

Contents:
---------

Contents
Description
EPG Merge
Get the source from git
Requirements
MySQL Setup
epgd Installation
epgd Configuration
Starting epgd and init-Scripts
Upgrade from older versions
MySQL Hints
Considerations


Description:
------------

epgd is part of the double team epgd+epg2vdr to effectively retrieve, store and import epg data to vdr. It is designed to handle large amount of data and pictures in a distributed environment with one epg-server and many possible vdr-clients - therefore it relays on mysql. 

Though it is possible to use epgd alone with mysql it only makes sense to use it as backend to the vdr-plugin epg2vdr. That being said you need to install, setup and configure mysql, epgd and epg2vdr in order to get a working environment.


EPG Merge:
----------

  The DVB and external events will be merged for the next 72 hours, DVB EPG is the main event provider which will 
  be enhanced by the external EPG and series data from constabel.net.
  If events can be merged, just starttime, duration and vps-flag of the DVB event will be used 
  from external provider. The epgd can mix events with up to 20 minutes difference of the start time
  depending on title and duration. If DVB and external EPG can't be merged (due to late program changes or sg.), 
  the DVB event will be placed in your EPG to avoid wrong events.
  Due to the EPG-merge the x-components delivered by DVB now also available. Notice that they only can handled for 
  DVB and merged events, not for events which only provided by the external source 
  The merge behavior is configured in the channlemap.conf, 0 disable the merge for the corresponding channel, 
  1 enables it and with merge 2 the provider ist hold in background and used as fall-back which is used in case 
  the merge 1 provider deliver poor data.


Get the source from git:
------------------------

get the source - probably done, when you reading this locally:
git clone git://projects.vdr-developer.org/vdr-epg-daemon.git


update the source:
cd /to/your/source/epgd
git pull

throwing away local changes and update to latest source:
cd /to/your/source/epgd
git checkout -f master
git reset --hard
git pull


Requirements:
-------------

  - libarchive
  - libcurl 7.10+
  - libxslt 1.1.24+
  - libxml2 
  - libmysql >= 5.07
  - libjansson4 libjansson-dev
  - libz
  - libssl-dev
  - libimlib2 libimlib2-dev

 Ubuntu (14.10):

  - libarchive12, libarchive-dev
  - libz, libz-dev
  - libssl-dev
  - libcurl4-nss-dev (oder libcurl3-nss-dev)
  - libjansson4 libjansson-dev
  - libxslt1.1 libxslt1-dev libxml2 libxml2-dev
  - libmysqlclient-dev libmysqlclient18
  - libjpeg-dev libjpeg
  - libimlib2 libimlib2-dev

My-SQL Setup:
-------------

You need a running mysql database. Consult the manual of your linux-flavour how to install and configure mysql. Remember to put your db-files in a large enough filesystem: with many channels, many pre fetched days and many pictures you will easily need 3 or more Gb of disk space. When mysql is up and running you need to prepare the database and access-rights for use with epgd:

- login as root: 
#> mysql -u root -p
 CREATE DATABASE epg2vdr charset utf8;
 CREATE USER 'epg2vdr'@'%' IDENTIFIED BY 'epg';
 GRANT ALL PRIVILEGES ON epg2vdr.* TO 'epg2vdr'@'%';

- login as epg2vdr for maintenance:
#> mysql -u epg2vdr -pepg -Depg2vdr
- or remote
#> mysql -u epg2vdr -pepg -Depg2vdr -h <host>

If you have problems with setting up and granting access rights to your epg2vdr user read "MySQL Hints" near the end of this file.

- show all users:
 use mysql
 SELECT * FROM user;


epgd Installation:
------------------

epgd is source distributed only. So you have to build your binary and make some adjustments to your system. So we try to help with some stuff in the contrib directory, it's not a bad idea to learn something about your linux's init-system and init-scripts.

- Unpack 
- Modify Make.config (BINDEST and PLGDEST) as you like
- Call "make"
- Call "make plugins"
- Call "make install"
- Call "make install-plugins"
- Create directory /etc/epgd
- cp configs/* /etc/epgd
- merge(*) the config parts of the epgd plugins (./PLUGINS/*/configs) to /etc/epgd
  -> merge /etc/epgd/channelmap.conf and /etc/epgd/epgd.conf
  -> copy ./PLUGINS/*/configs/*.xsl to /etc/epgd
- modify config (/etc/epgd/channelmap.conf and /etc/epgd/epgd.conf) like your needs
- Create the database (see below)
- Install epglv - the lib installed by make install and the funnctions created by epgd-tool if
   you use this for the initial database creation (if this not work look at epglv/README)
- Start epgd (see below)

If you don't know how to merge you can start of with 

#> cat ./PLUGINS/*/configs/channelmap.conf >>/etc/epgd/channelmap.conf
#> cat ./PLUGINS/*/configs/epgd.conf >>/etc/epgd/epgd.conf

This is only true if you use all plugins. If you use only one of them replace "*" with the plugin you use.

Look carefully through your generated files!


Starting epgd and init-Scripts:
-------------------------------

There are many ways to start epgd. For a first try or debugging sessions you may want to start it simply by typing at the root command-prompt (paths have to be adjusted according to your changes to BINDEST and PLGDEST in Make.config):

#> export LANG="de_DE.UTF-8"
#> ulimit -c unlimited #so you can torture the developers with back traces
#> /usr/local/bin/epgd -n -p /usr/local/lib/epgd/plugins

epgd is configured in /etc/epgd/epgd.conf. But you can overwrite the following options on the command line:

    -n              don't daemonize
    -t              log to stdout
    -c <config-dir> use config in <config-dir>
    -p <plugin-dir> load plugins from <plugin-dir>
    -l <log-level>  set log level

For production use you should start epgd after net- and mysql services via your init-system. Some start-scripts have been put to ./contrib hopefully serving your needs (or at least giving you an idea of how to proceed). 


epgd Configuration:
-------------------

  DbHost = 
    ip/name of mysql server host (default localhost)

  DbPort = 
    port of the mysql server (default 3306)

  DbName = 
    name of the database (default epg2vdr)

  DbUser = 
    database user (default epg2vdr)

  DbPass = 
    database password (default epg)

  DaysInAdvance = 
    (default 8)
    Download (insert) EPG information for the next # days

  DaysToUpdate = 
    (default 2)
    Update # already insert days 

  UpdateTime = 0
    Perform automatic update every # hours (default: 2 hours)
 
  XmlStoreToFs =
    Sorer XML files to the filesystem, for debugging (default 0)

  GetEpgImages = 
    (default 0)
    Download images with the EPG information

  MaxImagesPerEvent
    How many pictures per event should be downloaded

  EpgImageSize = 
    0 = 174x130
    1 = 329x245
    2 = 525x400

  CachePath = 
    path to cache the xml files if XmlStoreToFs or SeriesStoreToFs enabled (default /var/cache/epgd)

  SeriesEnabled
    get series from eplists.constabel.net (default 1) 

  SeriesPort
    (default 2006)

  SeriesStoreToFs
    (default 0)

  SeriesUrl
    (default eplists.constabel.net)

  ScrapEpg
    (default 1)
    scrap EPG content 0 / 1

  ScrapRecordings
    (default 1)
    scrap recordings 0 / 1

  EpgView 
    (default eventsview.sql)
    use this if you want to design your own view for detailed EPG views and live-plugin
    eventsview.sql can be overritten on updates, so use a different name
  
  CheckInitial = 1 (default 1)
    enable or disable the intitial update of your external EPG provider after starting epgd

  UpdateThreshold = 500 (default 200)
    call merging of DVB and external EPG after <x> DVB changes and update clients

  UseProxy
      Enable HTTP Proxy (default 0)

  HttpProxy
      Name of the proxy

  UserName
      Username of your proxy

  Password
      Password of your proxy

  Loglevel #
      Logging level 1-5 (Errors, Infos, Debug ...) (default 1)

When choosing a loglevel consider carefully the impact on i/o-performance of your system. If you are using some kind of pattern driven logfile parsing software like logchecker you may want to use ./contrib/epgd.ignore

Upgrade from older versions:
----------------------------

If you upgrade from an older version with different DB-API (as stored in the vdrs table) you can either start with a plain
database as discriped here or update youre database if explained in the release notes of the specific release.

  - stop VDR and EPGD
  - execute the old epgd-dropall script in your previous install dir (default /usr/local/bin/epgd-dropall)
  - delete your epg.data to have a clean EPG (default /var/cache/vdr/epg.data)
  - build and install epdg (make, make plugins, make install, make install-plugins)
  - start epgd
  - compile and install vdr-plugin-epg2vdr (see README in PLUGIN directory)


MYSQL HINTS:
-----------
  - If you cannot figure out why you get Access denied, remove all entries from the user table 
    that have Host values with wildcards contained (entries that match '%' or '_' characters). 
    A very common error is to insert a new entry with Host='%' and User='some_user', 
    thinking that this allows you to specify localhost to connect from the same machine. 
    The reason that this does not work is that the default privileges include an 
    entry with Host='localhost' and User=''. Because that entry has a Host value 'localhost' 
    that is more specific than '%', it is used in preference to the new entry when connecting 
    from localhost! The correct procedure is to insert a second entry with Host='localhost' 
    and User='some_user', or to delete the entry with Host='localhost' and User=''. 
    After deleting the entry, remember to issue a "FLUSH PRIVILEGES" statement to reload the grant tables.


  - *ATTENTION* if you have binary-logging (log_bin) enabled!
     disable it OR add log-bin-trust-function-creators to your configuration: 
       > echo "log-bin-trust-function-creators = 1" >> /etc/mysql/my.cnf
       > restart the database


Considerations:
---------------

By design epgd can handle multiple clients. If you looking for a solution for your vdr-network plan your implementation 
carefully. Which clients should be able to connect? Which epg-solution in in use on each client? Has any of the clients 
patches or plugins manipulating epg running? And now the most important hint: read the epg2vdr README before you continue.

epgd can generate large amounts of date, as well db-data as logging-data - so don't be surprised. 
Manipulating data (read, write, move around) does not only need appropriate disk space, 
but also appropriate I/O-performance.

If you run into problems and want to start from a clean base, stop epgd and drop all data 
(take a look at the scripts in ./scripts) and restart epgd.   
