Difference between revisions of "M4 Automatic CDR Export"

From Kolmisoft Wiki
Jump to navigationJump to search
 
(9 intermediate revisions by 2 users not shown)
Line 1: Line 1:
Automatic CDR Export allows to create periodic CDR Export tasks that will be executed automatically on configured time with specific query parameters. Exported data will be archived and sent via Email to User or specific Email address.
Automatic CDR Export allows to the creation of periodic CDR Export tasks that will be executed automatically on configured time with specific query parameters. Exported data will be archived and sent via Email to the User or specific Email address.


<br><br>
==CDR Export Templates==
==CDR Export Templates==


Automatic CDR Export requires to create at least one [[M2_CDR_Export_Templates|CDR Export Template]] so be sure to create one before proceeding.
Automatic CDR Export requires creating at least one [[M4_CDR_Export_Templates|CDR Export Template]] so be sure to create one before proceeding.


<br><br>
==Automatic CDR Export==
==Automatic CDR Export==


Line 13: Line 15:
[[File:M2 automatic cdr export list.png]]
[[File:M2 automatic cdr export list.png]]


In this list you may active/deactive CDR export tasks, edit or delete existing tasks and create new ones by pressing '''Add new Automatic CDR Export'''.
In this list, you may activate/deactivate CDR export tasks, edit or delete existing tasks and create new ones by pressing '''Add new Automatic CDR Export'''.


Note that '''Last run at''' and '''Next run at''' are shown in timezone defined by '''Time Zone''' setting.
Note that '''Last run at''' and '''Next run at''' are shown in the timezone defined by '''Time Zone''' setting.
 
<br><br>


==New Automatic CDR Export==
==New Automatic CDR Export==
Line 21: Line 25:
[[File:M2 new automatic cdr export.png]]
[[File:M2 new automatic cdr export.png]]


When creating new Automatic CDR Export you will see two sections: '''Query''' and '''Settings'''.
When creating new Automatic CDR Export you will see two sections: '''Query''' and '''Settings'''.


===Query section===
===Query section===


This section is very similar to filters in Last Calls page. You can set various filters for CDR Export query. These filters work in the same way as they work in Last Call page therefore you may test you query in Last Calls page and if you are happy with result, set same parameters in this Query section.
This section is very similar to filters on the Last Calls page. You can set various filters for CDR Export queries. These filters work in the same way as they work on the Last Call page, therefore, you may test your query on the Last Calls page and if you are happy with the result, set the same parameters in this Query section.


Note that date filters (From and Till) are missing. These filters depend on CDR Export '''Repeat period''' and will be described bellow.
Note that date filters (From and Till) are missing. These filters depend on CDR Export '''Repeat period''' and will be described below.


===Settings section===
===Settings section===
Line 33: Line 37:
====Name====
====Name====


Name is only for information purposes. This name is shown in Automatic CDR Export list.
The name is only for information purposes. This name is shown in the Automatic CDR Export list.


====Time Zone====
====Time Zone====


Time Zone is important setting and should be set correctly depending on situation. All other time related settings will use this Time Zone.
Time Zone is important setting and should be set correctly depending on situation. All other time-related settings will use this Time Zone.


For example, if you want to generate daily CDR Export to your User who lives in specific region, let’s say London, you should set timezone to '''GMT (+00:00) London'''. This way User will get CDR data starting from 00:00:00 till 23:59:59 '''in User’s timezone'''.
For example, if you want to generate daily CDR Export to your User who lives in a specific region, let’s say London, you should set the timezone to '''GMT (+00:00) London'''. This way User will get CDR data starting from 00:00:00 till 23:59:59 '''in User’s timezone'''.


Server and User may be in different timezones therefore it is important to set Time Zone to User’s timezone if you intend to generate CDR for this User.
Server and User may be in different timezones therefore it is important to set Time Zone to User’s timezone if you intend to generate CDR for this User.


Another thing to consider is when you set '''Start CDR Export at''' setting (will be described later). This variable will also depends on Time Zone setting. For example, maybe you server off-peer hours are 00:00:00 - 06:00:00, but in selected Time Zone off-peer hours may be different.
Another thing to consider is when you set '''Start CDR Export at''' setting (will be described later). This variable will also depend on Time Zone setting. For example, maybe you server off-peer hours are 00:00:00 - 06:00:00, but in selected Time Zone off-peer hours may be different.


====Repeat period====
====Repeat period====


Repeat period defines how often CDR Export is executed and which period is used for data retrieval. For this reason Query section is missing From and Till.
The repeat period defines how often CDR Export is executed and which period is used for data retrieval. For this reason Query section is missing From and Till.


Note that this setting depends on Time Zone variable.
Note that this setting depends on the Time Zone variable.


=====Hourly=====
=====Hourly=====
Line 61: Line 65:
=====Daily=====
=====Daily=====


CDR export will be executed each day. CDR data will correspond to whole '''previous''' day.  
CDR export will be executed each day. CDR data will correspond to the whole '''previous''' day.  


For example, if CDR export was executed on 2016-11-11 00:00:00, then CDR will be for 2016-11-10 00:00:00 - 2016-11-10 23:59:59 period.
For example, if CDR export was executed on 2016-11-11 00:00:00, then CDR will be for 2016-11-10 00:00:00 - 2016-11-10 23:59:59 period.
Line 67: Line 71:
=====Weekly=====
=====Weekly=====


CDR export will be executed each week. CDR data will correspond to whole '''previous''' week.  
CDR export will be executed each week. CDR data will correspond to the whole '''previous''' week.  


For example, if CDR export was executed on 2016-11-14 00:00:00 (monday), then CDR will be for 2016-11-07 00:00:00 (monday) - 2016-11-13 23:59:59 (sunday) period. Data period is always monday - sunday.
For example, if CDR export was executed on 2016-11-14 00:00:00 (Monday), then CDR will be for 2016-11-07 00:00:00 (Monday) - 2016-11-13 23:59:59 (Sunday) period. The data period is always Monday - Sunday.


=====Bi-weekly=====
=====Bi-weekly=====


CDR export will be executed each twice a month. CDR data will correspond either to 1-15 day of month or 16 - end of month.  
CDR export will be executed twice a month. CDR data will correspond either to 1-15 day of month or 16 - end of month.  


For example, if CDR export was executed on 2016-11-16 00:00:00, then CDR will be for 2016-11-01 00:00:00 - 2016-11-15 23:59:59 period. If CDR export was executed on 2016-12-01 00:00:00, then CDR will be for 2016-11-16 00:00:00 - 2016-11-30 23:59:59 period.  
For example, if CDR export was executed on 2016-11-16 00:00:00, then CDR will be for 2016-11-01 00:00:00 - 2016-11-15 23:59:59 period. If CDR export was executed on 2016-12-01 00:00:00, then CDR will be for 2016-11-16 00:00:00 - 2016-11-30 23:59:59 period.  
Line 79: Line 83:
=====Monthly=====
=====Monthly=====


CDR export will be executed each month. CDR data will correspond to whole '''previous''' month.  
CDR export will be executed each month. CDR data will correspond to the whole '''previous''' month.  


For example, if CDR export was executed on 2016-11-01 00:00:00, then CDR will be for 2016-10-01 00:00:00 - 2016-10-31 23:59:59 period. Data period is always first day of the month - last day of the month (whole month).
For example, if CDR export was executed on 2016-11-01 00:00:00, then CDR will be for 2016-10-01 00:00:00 - 2016-10-31 23:59:59 period. The data period is always the first day of the month - the last day of the month (whole month).


=====Only once=====
=====Only once=====


CDR export will be executed on specific date and time. Also, in Query section two additional filters will be displayed - From and Till. These filters allow to specify date period for CDR Export query (in selected Time Zone).
CDR export will be executed on a specific date and time. Also, in the Query section two additional filters will be displayed - From and Till. These filters allow specifying date period for CDR Export query (in selected Time Zone).


====Start CDR Export at====
====Start CDR Export at====


If you have selected '''Daily''', '''Weekly''', '''Bi-weekly''' or '''Monthly''' periods, you may select specific hours when Automatic CDR export should be executed. This is useful because you may choose off-peek hours. Automatic CDR Export is executed only during selected hour period.
If you have selected '''Daily''', '''Weekly''', '''Bi-weekly''' or '''Monthly''' periods, you may select specific hours when Automatic CDR export should be executed. This is useful because you may choose off-peek hours. Automatic CDR Export is executed only during a selected hour period.


If you have selected '''Only once''' period, you have to set full date (days and hours). This is because '''Only once''' is not periodic and should have specific date and time.
If you have selected '''Only once''' period, you have to set full date (days and hours). This is because '''Only once''' is not periodic and should have a specific date and time.


Things to consider:
Things to consider:


* Choose off-peek hours, this will reduce load on system;
* Choose off-peek hours, this will reduce the load on the system;
* Be carefull when midnight is selected (00:00) because User may start new Call just before midnight and that Call may last for more than an hour. In this case, you may want to start CDR export few hours after midnight so all User’s Calls have time to finish;
* Be careful when midnight is selected (00:00) because the User may start a new Call just before midnight and that Call may last for more than an hour. In this case, you may want to start CDR export a few hours after midnight so all User’s Calls have time to finish;
* Hourly period may be inaccurate for the reason described above.
* Hourly period may be inaccurate for the reason described above.


Note that this setting depends on Time Zone variable.
Note that this setting depends on the Time Zone variable.


====Template====
====Template====


Select which [[M2_CDR_Export_Templates|CDR Export Template]] to use.
Select which [[M4_CDR_Export_Templates|CDR Export Template]] to use.


====Send CDR to====
====Send CDR to====
Line 111: Line 115:
Also, make sure that email sending is configured correctly.
Also, make sure that email sending is configured correctly.


CDRs will be '''archived''' and sent to User/Email.  
CDRs will be '''archived''' and sent to the User/Email.  


If an archive is sent successfully, the User should receive a success email which can be customized in [[Email_variables#CDR_Export|Emails]] settings. CDR archive will be attached.
If an archive is sent successfully, the User should receive a success email which can be customized in [[Email_variables#CDR_Export|Emails]] settings. CDR archive will be attached.


'''Note:''' archive size is limited to '''10MB'''! If the CDR archive is larger, then the archive will not be sent via Email and User will receive error Email instead.
'''Note:''' archive size is limited to '''10MB'''! If the CDR archive is larger, then the archive will not be sent via Email and User will receive error Email instead.
'''Note:''' if '''Custom Email''' option is yous and you want to send CDRs to more than one email, please separate the emails with ''';''' sign.


'''FTP''' - Please fill FTP server details at [https://wiki.kolmisoft.com/index.php/M2-M4_Settings#FTP FTP] tab.
'''FTP''' - Please fill FTP server details at [https://wiki.kolmisoft.com/index.php/M2-M4_Settings#FTP FTP] tab.
'''SFTP''' - Please fill SFTP server details at [https://wiki.kolmisoft.com/index.php/M2-M4_Settings#SFTP SFTP] tab.
<br><br>


==Next run at==
==Next run at==


In Automatic CDR Export list you should see '''Next run at''' column. This shows date and time (in selected Timezone) when CDR export will be executed.  
In the Automatic CDR Export list, you should see '''Next run at''' column. This shows the date and time (in the selected Timezone) when CDR export will be executed.  


What happens when you create new Automatic CDR Export? When CDR Export will be executed? It depends on Repeat period. New Automatic CDR Export is executed '''on next period'''. For example, if you have created '''Hourly''' Automatic CDR Export on 2016-11-11 09:11:45, then the first time CDR Export is executed will be on 2016-11-11 10:00:00.
What happens when you create new Automatic CDR Export? When CDR Export will be executed? It depends on the Repeat period. New Automatic CDR Export is executed '''on next period'''. For example, if you have created '''Hourly''' Automatic CDR Export on 2016-11-11 09:11:45, then the first time CDR Export is executed will be on 2016-11-11 10:00:00.


If you have created '''Daily''' Automatic CDR Export on 2016-11-11 09:11:45, then the first time CDR Export is executed will be on 2016-11-12 00:00:00.
If you have created '''Daily''' Automatic CDR Export on 2016-11-11 09:11:45, then the first time CDR Export is executed will be on 2016-11-12 00:00:00.
Line 129: Line 139:
If you have created '''Monthly''' Automatic CDR Export on 2016-11-11 09:11:45, then the first time CDR Export is executed will be on 2016-12-01 00:00:00.
If you have created '''Monthly''' Automatic CDR Export on 2016-11-11 09:11:45, then the first time CDR Export is executed will be on 2016-12-01 00:00:00.


Of course hours may be different depending on '''Start CDR Export at''', but the idea is that CDR Export is executed on next period, either it is next hour, next day, next week or next month.
Of course, hours may be different depending on '''Start CDR Export at''', but the idea is that CDR Export is executed on the next period, either it is the next hour, next day, next week, or next month.


<br><br>
==File names==
==File names==


CDR Export output is a CSV file compressed in tgz archive. Filename is based on dates, for example if CDR Export was executed for 2016-11-01 00:00:00 - 2016-11-30 23:59:59 period, then file names will be:
CDR Export output is a CSV file compressed in tgz archive. The filename is based on dates, for example, if CDR Export was executed for 2016-11-01 00:00:00 - 2016-11-30 23:59:59 period, then file names will be:


'''CSV filename:''' M2_CDR_export_20161122_000000_20161122_235959.csv<br>
'''CSV filename:''' M2_CDR_export_20161122_000000_20161122_235959.csv<br>
'''TGZ filename:''' M2_CDR_export_20161122_000000_20161122_235959.tgz
'''TGZ filename:''' M2_CDR_export_20161122_000000_20161122_235959.tgz


<br><br>
==CDR Export and performance==
==CDR Export and performance==


CDR Export is resource intensive task. Consider having as little Automatic CDR Export tasks as possible. Also shorter Repeat periods have better peformance.
CDR Export is resource intensive task. Consider having as few Automatic CDR Export tasks as possible. Also shorter Repeat periods have better performance.


It is advisable to set '''Start CDR Export at''' to off-peak hours. Also,''' Start CDR Export at''' allows to spread tasks across hours during the day.
It is advisable to set '''Start CDR Export at''' to off-peak hours. Also,''' Start CDR Export at''' allows spreading tasks across hours during the day.


<br><br>
==How old CDR Export tasks are handled==
==How old CDR Export tasks are handled==


There might be situations when script responsible for CDR Export functionality was not running for long time. What happens to all CDRs that were supposed to be generated during that time when script was not running? Only single CDR for latest period will be generated. This is done intentionally to prevent high amount of CDR Export tasks to run at a time.
There might be situations when the script responsible for CDR Export functionality was not running for a long time. What happens to all CDRs that were supposed to be generated during that time when the script was not running? Only a single CDR for the latest period will be generated. This is done intentionally to prevent a high amount of CDR Export tasks to run at a time.


For example, if '''Monthly''' CDR Export tasks was created on January but for some reason CDR Export script was not executed until November, then CDR Export for January-September will be skipped. Script will generate CDR Export for October and sets '''Next run at''' to December.
For example, if '''Monthly''' CDR Export tasks were created in January but for some reason the CDR Export script was not executed until November, then CDR Export for January-September will be skipped. The script will generate CDR Export for October and will set '''Next run at''' to December.


This logic does not apply to '''Only once''' period. All tasks with '''Only once''' period will be executed.
This logic does not apply to '''Only once''' period. All tasks with '''Only once''' period will be executed.
<br><br>


==See also==
==See also==


[[M2_CDR_Export_Templates|CDR Export Templates]]<br>
* [[M4_CDR_Export_Templates|CDR Export Templates]]<br>
[[Email_variables#CDR_Export|Emails Templates variables]]
* [[Email_variables#CDR_Export|Emails Templates variables]]

Latest revision as of 08:03, 7 May 2024

Automatic CDR Export allows to the creation of periodic CDR Export tasks that will be executed automatically on configured time with specific query parameters. Exported data will be archived and sent via Email to the User or specific Email address.



CDR Export Templates

Automatic CDR Export requires creating at least one CDR Export Template so be sure to create one before proceeding.



Automatic CDR Export

Automatic CDR export is configured in BILLING -> CDR -> Export -> Automatic

Here you will find a list of already configured Automatic CDR Exports:

M2 automatic cdr export list.png

In this list, you may activate/deactivate CDR export tasks, edit or delete existing tasks and create new ones by pressing Add new Automatic CDR Export.

Note that Last run at and Next run at are shown in the timezone defined by Time Zone setting.



New Automatic CDR Export

M2 new automatic cdr export.png

When creating new Automatic CDR Export you will see two sections: Query and Settings.

Query section

This section is very similar to filters on the Last Calls page. You can set various filters for CDR Export queries. These filters work in the same way as they work on the Last Call page, therefore, you may test your query on the Last Calls page and if you are happy with the result, set the same parameters in this Query section.

Note that date filters (From and Till) are missing. These filters depend on CDR Export Repeat period and will be described below.

Settings section

Name

The name is only for information purposes. This name is shown in the Automatic CDR Export list.

Time Zone

Time Zone is important setting and should be set correctly depending on situation. All other time-related settings will use this Time Zone.

For example, if you want to generate daily CDR Export to your User who lives in a specific region, let’s say London, you should set the timezone to GMT (+00:00) London. This way User will get CDR data starting from 00:00:00 till 23:59:59 in User’s timezone.

Server and User may be in different timezones therefore it is important to set Time Zone to User’s timezone if you intend to generate CDR for this User.

Another thing to consider is when you set Start CDR Export at setting (will be described later). This variable will also depend on Time Zone setting. For example, maybe you server off-peer hours are 00:00:00 - 06:00:00, but in selected Time Zone off-peer hours may be different.

Repeat period

The repeat period defines how often CDR Export is executed and which period is used for data retrieval. For this reason Query section is missing From and Till.

Note that this setting depends on the Time Zone variable.

Hourly

CDR export will be executed each hour. CDR data will correspond to whole previous hour.

For example, if CDR export was executed on 2016-11-11 10:00:00, then CDR will be for 2016-11-11 09:00:00 - 2016-11-11 09:59:59 period.

In other words, it will work as if you set From to 2016-11-11 09:00:00 and Till to 2016-11-11 09:59:59 in date filters.

Daily

CDR export will be executed each day. CDR data will correspond to the whole previous day.

For example, if CDR export was executed on 2016-11-11 00:00:00, then CDR will be for 2016-11-10 00:00:00 - 2016-11-10 23:59:59 period.

Weekly

CDR export will be executed each week. CDR data will correspond to the whole previous week.

For example, if CDR export was executed on 2016-11-14 00:00:00 (Monday), then CDR will be for 2016-11-07 00:00:00 (Monday) - 2016-11-13 23:59:59 (Sunday) period. The data period is always Monday - Sunday.

Bi-weekly

CDR export will be executed twice a month. CDR data will correspond either to 1-15 day of month or 16 - end of month.

For example, if CDR export was executed on 2016-11-16 00:00:00, then CDR will be for 2016-11-01 00:00:00 - 2016-11-15 23:59:59 period. If CDR export was executed on 2016-12-01 00:00:00, then CDR will be for 2016-11-16 00:00:00 - 2016-11-30 23:59:59 period.

Monthly

CDR export will be executed each month. CDR data will correspond to the whole previous month.

For example, if CDR export was executed on 2016-11-01 00:00:00, then CDR will be for 2016-10-01 00:00:00 - 2016-10-31 23:59:59 period. The data period is always the first day of the month - the last day of the month (whole month).

Only once

CDR export will be executed on a specific date and time. Also, in the Query section two additional filters will be displayed - From and Till. These filters allow specifying date period for CDR Export query (in selected Time Zone).

Start CDR Export at

If you have selected Daily, Weekly, Bi-weekly or Monthly periods, you may select specific hours when Automatic CDR export should be executed. This is useful because you may choose off-peek hours. Automatic CDR Export is executed only during a selected hour period.

If you have selected Only once period, you have to set full date (days and hours). This is because Only once is not periodic and should have a specific date and time.

Things to consider:

  • Choose off-peek hours, this will reduce the load on the system;
  • Be careful when midnight is selected (00:00) because the User may start a new Call just before midnight and that Call may last for more than an hour. In this case, you may want to start CDR export a few hours after midnight so all User’s Calls have time to finish;
  • Hourly period may be inaccurate for the reason described above.

Note that this setting depends on the Time Zone variable.

Template

Select which CDR Export Template to use.

Send CDR to

Select who should receive CDR Export. It can be either User, a specific Email address, or an FTP server. If User is selected then the Email address will be taken from Users settings (make sure that User has email).

Also, make sure that email sending is configured correctly.

CDRs will be archived and sent to the User/Email.

If an archive is sent successfully, the User should receive a success email which can be customized in Emails settings. CDR archive will be attached.

Note: archive size is limited to 10MB! If the CDR archive is larger, then the archive will not be sent via Email and User will receive error Email instead.

Note: if Custom Email option is yous and you want to send CDRs to more than one email, please separate the emails with ; sign.

FTP - Please fill FTP server details at FTP tab.

SFTP - Please fill SFTP server details at SFTP tab.



Next run at

In the Automatic CDR Export list, you should see Next run at column. This shows the date and time (in the selected Timezone) when CDR export will be executed.

What happens when you create new Automatic CDR Export? When CDR Export will be executed? It depends on the Repeat period. New Automatic CDR Export is executed on next period. For example, if you have created Hourly Automatic CDR Export on 2016-11-11 09:11:45, then the first time CDR Export is executed will be on 2016-11-11 10:00:00.

If you have created Daily Automatic CDR Export on 2016-11-11 09:11:45, then the first time CDR Export is executed will be on 2016-11-12 00:00:00.

If you have created Monthly Automatic CDR Export on 2016-11-11 09:11:45, then the first time CDR Export is executed will be on 2016-12-01 00:00:00.

Of course, hours may be different depending on Start CDR Export at, but the idea is that CDR Export is executed on the next period, either it is the next hour, next day, next week, or next month.



File names

CDR Export output is a CSV file compressed in tgz archive. The filename is based on dates, for example, if CDR Export was executed for 2016-11-01 00:00:00 - 2016-11-30 23:59:59 period, then file names will be:

CSV filename: M2_CDR_export_20161122_000000_20161122_235959.csv
TGZ filename: M2_CDR_export_20161122_000000_20161122_235959.tgz



CDR Export and performance

CDR Export is resource intensive task. Consider having as few Automatic CDR Export tasks as possible. Also shorter Repeat periods have better performance.

It is advisable to set Start CDR Export at to off-peak hours. Also, Start CDR Export at allows spreading tasks across hours during the day.



How old CDR Export tasks are handled

There might be situations when the script responsible for CDR Export functionality was not running for a long time. What happens to all CDRs that were supposed to be generated during that time when the script was not running? Only a single CDR for the latest period will be generated. This is done intentionally to prevent a high amount of CDR Export tasks to run at a time.

For example, if Monthly CDR Export tasks were created in January but for some reason the CDR Export script was not executed until November, then CDR Export for January-September will be skipped. The script will generate CDR Export for October and will set Next run at to December.

This logic does not apply to Only once period. All tasks with Only once period will be executed.



See also