Difference between revisions of "Rate import from CSV"

From Kolmisoft Wiki
Jump to navigationJump to search
(42 intermediate revisions by 10 users not shown)
Line 16: Line 16:
*  CSV Column Separator
*  CSV Column Separator
*  CSV Decimal Separator
*  CSV Decimal Separator
 
<br><br>
 
[[File:Settings-functionality.png]]
[[File:Settings_functionality_tab_csv_settings.png]]
<br><br>
 
 
File and CSV separator settings should match. If they do not, you will get errors when trying to import file.
File and CSV separator settings should match. If they do not, you will get errors when trying to import file.


* [[Good CSV rate file examples]]
* [[Good CSV rate file examples]]
* [[Bad CSV rate file examples]]
* [[Bad CSV rate file examples]]
 
<br><br>
=== Where can I get an example file with available prefixes? ===
=== Where can I get an example file with available prefixes? ===


Go to '''SETTINGS -> Billing -> Tariffs''' and click on [[Image:icon_excel.png]] '''Destinations CSV'''.
Go to '''SETTINGS -> Billing -> Tariffs''' and click on [[Image:icon_excel.png]] '''Destinations CSV'''.
<br><br>
<br><br>
= Description =
= Description =


Line 50: Line 45:


'''IMPORTANT NOTE: When importing, prefixes should be in E.164 format!!! For example, no international prefixes!'''
'''IMPORTANT NOTE: When importing, prefixes should be in E.164 format!!! For example, no international prefixes!'''
 
<br><br>
 
 
== 1. File upload ==
== 1. File upload ==


Line 58: Line 51:


Then click on '''Browse''' and select CSV file with rates. Click '''Upload file''':
Then click on '''Browse''' and select CSV file with rates. Click '''Upload file''':
 
<br><br>
[[Image:rateimportfromcsv1.png]]
[[Image:rateimportfromcsv1.png]]
 
<br><br>
 
== 2.1 Column assignment - wrong separators detected ==
== 2.1 Column assignment - wrong separators detected ==
Sometimes the system is able to detect that CSV separators are wrong and suggests you to change them:
Sometimes the system is able to detect that CSV separators are wrong and suggests you to change them:
Line 70: Line 62:


The file is uploaded and the first five lines of the file are available to check:
The file is uploaded and the first five lines of the file are available to check:
 
<br><br>
[[Image:rateimportfromcsv2.png]]
[[Image:rateimportfromcsv2.png]]
 
<br><br>
This helps us in the next step, to assign CSV file's columns to MOR's variables. We have the following window:
This helps us in the next step, to assign CSV file's columns to MOR's variables. We have the following window:
<br><br>
[[Image:rateimportfromcsv3_2.png]]
<br><br>
Here we must assign columns from the CSV file. '''Prefix''' and '''Rate''' are mandatory fields. All other fields can be empty. If during import system will find corresponding headers in CSV file, columns will be assigned automatically. You just need to check is it correct.


[[Image:rateimportfromcsv3.png]]
'''[[Rate Effective From | Effective from]]''' date can be imported in various formats of date. If there is no Effective From date in CSV file you are able to set that date manually, otherwise Default value (it is now) will be taken.


Here we must assign columns from the CSV file. '''Prefix''' and '''Rate''' are mandatory fields. All other fields can be empty.
In addition, you can also use of one of the '''quick''' buttons to set the time to '''Now''' (this will give you a full current time, e.g. 2015-09-29 19:06:21) or '''Start of today''' (e.g. 2015-09-29 00:00:00).
'''
Connection Fee''', '''Increment''' and '''Min. Time''' have their defaults. If you wish, you can enter values manually  for them. Manually entered values have the highest priority. If you omit them, then MOR inserts values from the CSV file. If there is no assigned column for these fields, then default values will be added to database (DB).


There are a group of values under the name '''USA specifics'''. These are only for USA/Canada. Users from these countries need to use these fields with their CSV files.
'''Connection Fee''', '''Increment''' and '''Min. Time''' have their defaults. If you wish, you can enter values manually  for them. Manually entered values have the highest priority. If you omit them, then MOR inserts values from the CSV file. If there is no assigned column for these fields, then default values will be added to database (DB).


In the following example, we select fields which match our CSV file. Also entered are some manual values for '''Increment''' and '''Min.Time'''. We  count time by 30s increments and minimum time for the call is 60s, with these settings (you can put any values you want/need):
You can also import '''Ghost Percent''' with the rates. It works the same way as the '''Ghost minute percent''' in the [[Calling_Card_Groups | Calling Cards Groups]]: it is a value to trick the user into thinking he has more minutes when he really has less.


[[Image:rateimportfromcsv4.png]]
'''Action''' - If this column have keyword DELETE, corresponding rate (prefix) is deleted if already exists in the tariff. If such rate does not exist in the system it will be ignored (not imported). All other values except DELETE in Action column are ignored.


This way of selecting CSV rows and assigning them to MOR values lets us import various CSV file formats.
This way of selecting CSV rows and assigning them to MOR values lets us import various CSV file formats.
 
<br><br>
 
===Additional options===
=== Rate Day Type ===
<br><br>
 
[[File:Importfromcsv.png]]
''This option is available starting from [[MOR_9_documentation | MOR 9]]''
<br><br>
 
'''Rate Day Type'''<br>
<!---This option is available starting from [[MOR_9_documentation | MOR 9]]--->
It is possible to select to which Day Type rates apply before importing them.
It is possible to select to which Day Type rates apply before importing them.


So in two steps it is possible to import separate rates for Work Days and Free Days, or (in a single step) to import rates for All Days at once.
So in two steps it is possible to import separate Rates for Work Days and Free Days, or (in a single step) to import rates for All Days at once.


<br>
'''On/Off Peak Tariff import'''<br>
[[Image:rate_day_type.png]]
<!---This option is available starting from [[MOR_10_documentation | MOR 10]]--->
<br><br>
If you need to import tariffs for on/off peak periods, all you need to do is upload another tariff on top of the one you have with a different  
 
=== On/Off Peak Tariff import ===
 
''This option is available starting from [[MOR_10_documentation | MOR 10]]''
 
If you need to import tariffs for on/off peak periods, all you need to do is upload another tariff on top of the one you have with a different<br>
time period.
time period.
[[Image:rate_day_type.png]]
* You are now able to specify peak and off-peak periods (hours, minutes, seconds) when importing tariffs for work days and weekdays.
* You are now able to specify peak and off-peak periods (hours, minutes, seconds) when importing tariffs for work days and weekdays.
* Important notice - time periods cannot cover each other.
* Important notice - time periods cannot cover each other.
* If for example you upload a tariff from 8am to 12pm. The Tariff that is uploaded on top can only be from 12pm to 8am.
* If for example you upload a tariff from 8am to 12pm. The Tariff that is uploaded on top can only be from 12pm to 8am.
<br><br>
=== Destination name update on rate import ===
''This option is available starting from [[MOR_9_documentation | MOR 9]]''
<br><br>
When this option is selected destination name is updated from csv file during rate import
<br><br>
[[File:Update_destinations_from_file.png]]
<br><br>


'''Update Destination Names from file'''<br>


=== Updating Directions for Destinations based on Country Code ===
When this option is selected destination name is updated from csv file during rate import.


''This option is available starting from [[MOR 12]]''
'''Delete Rates for Prefixes which are not present in imported file'''


This option allows you to delete existing rates, which will not be updated.(The prefixes that belong to these rates are not a part of the file being imported).


This functionality lets you reassign Destinations to different Directions by Country Codes specified on CSV file. When ''Update Directions for Destinations based on Country Code:'' is selected country code field is necessary.
'''Warn about Rates greater than'''<br>
This option is is used in Tariff analysis to warn about Rates which are greater than a certain value. This check is disabled by entering 0. '''Note''': The setting does not affect Rate import in any way since it is only used for informational purposes.


[[File:Importfromcsv.png]]
'''Ignore Effective from time when updating Rates'''<br>
When this option is selected only the date part of Effective from value is matched when updating Rates. The option is useful when the CSV file with rates contains only date as Effective from.  '''Note''': when there are several rates for the same date and this options is enabled, all the rates for this date will be updated.
<br><br>


== 3. Column confirmation ==
== 3. Column confirmation ==


The next step is to review our selection and confirm that everything is correct. We can go Back or forward to the next step:
The next step is to review our selection and confirm that everything is correct. We can go Back or forward to the next step:
 
<br><br>
[[Image:rateimportfromcsv5.png]]
[[Image:rateimportfromcsv5.png]]
 
<br><br>
 
'''NOTE''' Steps where is nothing to do  (value = 0) will be skipped.
<br><br>


== 4. Analysis ==
== 4. Analysis ==
Line 147: Line 129:
It will take some time to accomplish this step. MOR checks many things from DB and CSV files: it compares rates, searches for duplicates, fixes small errors and so on. After the analysis is completed, we get something like this :
It will take some time to accomplish this step. MOR checks many things from DB and CSV files: it compares rates, searches for duplicates, fixes small errors and so on. After the analysis is completed, we get something like this :
<br><br>
<br><br>
[[Image:rateimportfromcsv6.png]]
[[Image:Rate_import_analize.png]]
<br><br>
<br><br>
The most interesting line is '''9 bad destination(s)'''. We click on the List to check them. In the new window, we can see which destinations are bad and we can decide what to do with them. Is it possible that there are no directions for them, MOR doesn't know to which country to assign this destination, or maybe these destinations have duplicates in CSV file? You have to fix this if you want to import the whole file. Here you can stop and fix your CSV file, or you can ignore these errors and continue.
The most interesting line is '''737 bad destination(s)'''. We click on the List to check them. In the new window, we can see which destinations are bad and we can decide what to do with them. Is it possible that there are no directions for them, MOR doesn't know to which country to assign this destination, or maybe these destinations have duplicates in CSV file? You have to fix this if you want to import the whole file. Here you can stop and fix your CSV file, or you can ignore these errors and continue.<br>
 
Line '''Zero Rates in CSV file''' displays number of rates wich has zero value. These rates will not be ignored, and will be imported.<br>
Line '''High Rates''' displays a number of rates which are greater than a value set in '''Warn about Rates greater than''' setting. These Rates will still be imported.<br>
We continue the demo by clicking ''Proceed''.
We continue the demo by clicking ''Proceed''.


The following steps are separated to avoid excessive demands on processing power if a rate file is huge. This keeps the browser from crashing and allows it to successfully import 160.000+ destinations from one CSV file (this has been tested).
The following steps are separated to avoid excessive demands on processing power if a rate file is huge. This keeps the browser from crashing and allows it to successfully import 160.000+ destinations from one CSV file (this has been tested).
<br><br>


== 5. Creating destinations ==
== 5.1. Creating Destinations ==


MOR creates new destinations here. These destinations are necessary for the rates, which will be added in step 7. This window is just informative, no decisions here:
MOR creates new destinations here. These Destinations are necessary for the Rates, which will be added in step 7. This window is just informative, no decisions here:


MOR creates only Destinations for prefixes which are not yet in MOR's DB and determines whether it is possible to find out the Direction (Country) for these new prefixes (Destinations).
MOR creates only Destinations for prefixes which are not yet in MOR's DB and determines whether it is possible to find out the Direction (Country) for these new prefixes (Destinations).
Line 165: Line 149:
1. The system checks the available prefixes for the best match and selects found Direction as a basis for the new prefix (Destination).
1. The system checks the available prefixes for the best match and selects found Direction as a basis for the new prefix (Destination).


For example: if you try to import the rate for prefix 370630 and this prefix is unknown to MOR, MOR checks the longest matching prefix - 3706 in this case - and finds out that it is Lithuania MOB. So it creates a new Destination with Prefix 370630 in which the Direction will be Lithuania and the subcode MOB (mobile network).
For example: if you try to import the rate for prefix 370630 and this prefix is unknown to MOR, MOR checks the longest matching prefix - 3706 in this case - and finds out that it is Lithuania Mobile. So it creates a new Destination with Prefix 370630 in which the Direction will be Lithuania.


2. If Country code is available in the imported file, MOR assigns this Direction to the newly created Destination.
2. If Country code is available in the imported file, MOR assigns this Direction to the newly created Destination.


For example: if again you try to import 370630 and it is unknown to MOR, but you have LTU in same CSV line for Country Code field, MOR will create such a Destination with Direction = Lithuania.
For example: if again you try to import 370630 and it is unknown to MOR, but you have LTU in same CSV line for Country Code field, MOR will create such a Destination with Direction = Lithuania.
 
<br><br>
 
[[Image:rateimportfromcsv7.png]]
[[Image:rateimportfromcsv7.png]]
<br><br>
'''NOTE''' Because Destination cannot be empty (space), during the Destination creation, MOR automatically sets Destination name as Direction, Destination name will be Direction.
<br><br>
== 5.2. Updating Destinations ==


Step appears when '''Update Destination Names from file''' is check in second step.
<br><br>
== 6. Updating rates ==
== 6. Updating rates ==
To update existing rates your CSV file should contain prefixes that already exist in Tariff. Also, it should contain the exact ''Effective from'' date. If your CSV file does not contain Effective from date then you can set it manually in Step 2, just make sure that date is the same as in Tariff. Only then rates will be updated. If there are no existing rates with the exact prefix and effective from then new rates will be created.<br>


Here MOR updates rates with the values from CSV files. These rates already exist in DB. Just click ''Proceed'':
Here MOR updates rates with the values from CSV files. These rates already exist in DB. Just click ''Proceed'':
 
<br><br>
[[Image:rateimportfromcsv8.png]]
[[Image:rateimportfromcsv8.png]]
<br><br>


== 7. Creating new rates ==
== 7. Creating new rates ==


Finally, MOR creates all rates and we are all done!
Here, MOR creates rates that have not existed previously. Click ''Proceed'' afterwards.
 
<br><br>
[[Image:rateimportfromcsv9.png]]
[[Image:rateimportfromcsv9.png]]
<br><br>
== 8. Deleting rates ==


This is the final step. MOR deletes the rates which were not a part of the file being imported.
You're all done!


== NOTES ==
== NOTES ==
Line 206: Line 202:


More info: [[Integrity Check]]
More info: [[Integrity Check]]
 
<br><br>
= Example Videos =
= Example Videos =


<flashow>http://wiki.kolmisoft.com/videos/rates_import_from_csv.swf|width=600|height=455</flashow>
<flashow>http://wiki.kolmisoft.com/videos/rates_import_from_csv.swf|width=600|height=455</flashow>




Line 216: Line 211:


<flashow>http://wiki.kolmisoft.com/videos/tariff_import_from_csv.swf|width=800|height=618</flashow>
<flashow>http://wiki.kolmisoft.com/videos/tariff_import_from_csv.swf|width=800|height=618</flashow>
<br><br>
Another example:
<html>
<iframe width="640" height="360" src="http://www.youtube.com/embed/XzqORQHpgpo" frameborder="0"></iframe>
</html>


Another example with Provider tariff:


<html>
<iframe width="640" height="360" src="http://www.youtube.com/embed/OVlrAJ5_a44" frameborder="0"></iframe>
</html>


= See also =
= See also =


* [[Importing Tariffs from CSV with wrong Regional Settings]]
* [[Good CSV rate file examples]]
* [[Good CSV rate file examples]]
* [[Bad CSV rate file examples]]
* [[Bad CSV rate file examples]]
* [[How to create a csv file]]
* [[How to create a csv file]]

Revision as of 13:46, 15 February 2018

File format

File should be CSV (comma separated values).

Necessary fields in CSV line:

  • Prefix
  • Price

No other fields are necessary.

First line can be header.

Value Separator and Decimal Separator MUST be set correctly in SETTINGS -> Setup -> Settings -> Functionality:

  • CSV Column Separator
  • CSV Decimal Separator



Settings-functionality.png

File and CSV separator settings should match. If they do not, you will get errors when trying to import file.



Where can I get an example file with available prefixes?

Go to SETTINGS -> Billing -> Tariffs and click on Icon excel.png Destinations CSV.

Description

It is very tedious work to add several thousand rates by hand, so you can import them from a CSV file.

There are seven steps for importing rates from a CSV file:

  1. File upload
  2. Column assignment
  3. Column confirmation
  4. Analysis
  5. Creating destinations
  6. Updating rates
  7. Creating new rates

We will cover each of these one by one.

IMPORTANT NOTE: When importing, prefixes should be in E.164 format!!! For example, no international prefixes!

1. File upload

Select Excel.png Import from CSV for the Tariff you want to import rates.

Then click on Browse and select CSV file with rates. Click Upload file:

Rateimportfromcsv1.png

2.1 Column assignment - wrong separators detected

Sometimes the system is able to detect that CSV separators are wrong and suggests you to change them:

Tariff import wrong separators.png

2.2 Column assignment

The file is uploaded and the first five lines of the file are available to check:

Rateimportfromcsv2.png

This helps us in the next step, to assign CSV file's columns to MOR's variables. We have the following window:

Rateimportfromcsv3 2.png

Here we must assign columns from the CSV file. Prefix and Rate are mandatory fields. All other fields can be empty. If during import system will find corresponding headers in CSV file, columns will be assigned automatically. You just need to check is it correct.

Effective from date can be imported in various formats of date. If there is no Effective From date in CSV file you are able to set that date manually, otherwise Default value (it is now) will be taken.

In addition, you can also use of one of the quick buttons to set the time to Now (this will give you a full current time, e.g. 2015-09-29 19:06:21) or Start of today (e.g. 2015-09-29 00:00:00).

Connection Fee, Increment and Min. Time have their defaults. If you wish, you can enter values manually for them. Manually entered values have the highest priority. If you omit them, then MOR inserts values from the CSV file. If there is no assigned column for these fields, then default values will be added to database (DB).

You can also import Ghost Percent with the rates. It works the same way as the Ghost minute percent in the Calling Cards Groups: it is a value to trick the user into thinking he has more minutes when he really has less.

Action - If this column have keyword DELETE, corresponding rate (prefix) is deleted if already exists in the tariff. If such rate does not exist in the system it will be ignored (not imported). All other values except DELETE in Action column are ignored.

This way of selecting CSV rows and assigning them to MOR values lets us import various CSV file formats.

Additional options



Importfromcsv.png

Rate Day Type
It is possible to select to which Day Type rates apply before importing them.

So in two steps it is possible to import separate Rates for Work Days and Free Days, or (in a single step) to import rates for All Days at once.

On/Off Peak Tariff import
If you need to import tariffs for on/off peak periods, all you need to do is upload another tariff on top of the one you have with a different time period.

  • You are now able to specify peak and off-peak periods (hours, minutes, seconds) when importing tariffs for work days and weekdays.
  • Important notice - time periods cannot cover each other.
  • If for example you upload a tariff from 8am to 12pm. The Tariff that is uploaded on top can only be from 12pm to 8am.

Update Destination Names from file

When this option is selected destination name is updated from csv file during rate import.

Delete Rates for Prefixes which are not present in imported file

This option allows you to delete existing rates, which will not be updated.(The prefixes that belong to these rates are not a part of the file being imported).

Warn about Rates greater than
This option is is used in Tariff analysis to warn about Rates which are greater than a certain value. This check is disabled by entering 0. Note: The setting does not affect Rate import in any way since it is only used for informational purposes.

Ignore Effective from time when updating Rates
When this option is selected only the date part of Effective from value is matched when updating Rates. The option is useful when the CSV file with rates contains only date as Effective from. Note: when there are several rates for the same date and this options is enabled, all the rates for this date will be updated.

3. Column confirmation

The next step is to review our selection and confirm that everything is correct. We can go Back or forward to the next step:

Rateimportfromcsv5.png

NOTE Steps where is nothing to do (value = 0) will be skipped.

4. Analysis

It will take some time to accomplish this step. MOR checks many things from DB and CSV files: it compares rates, searches for duplicates, fixes small errors and so on. After the analysis is completed, we get something like this :

Rate import analize.png

The most interesting line is 737 bad destination(s). We click on the List to check them. In the new window, we can see which destinations are bad and we can decide what to do with them. Is it possible that there are no directions for them, MOR doesn't know to which country to assign this destination, or maybe these destinations have duplicates in CSV file? You have to fix this if you want to import the whole file. Here you can stop and fix your CSV file, or you can ignore these errors and continue.
Line Zero Rates in CSV file displays number of rates wich has zero value. These rates will not be ignored, and will be imported.
Line High Rates displays a number of rates which are greater than a value set in Warn about Rates greater than setting. These Rates will still be imported.
We continue the demo by clicking Proceed.

The following steps are separated to avoid excessive demands on processing power if a rate file is huge. This keeps the browser from crashing and allows it to successfully import 160.000+ destinations from one CSV file (this has been tested).

5.1. Creating Destinations

MOR creates new destinations here. These Destinations are necessary for the Rates, which will be added in step 7. This window is just informative, no decisions here:

MOR creates only Destinations for prefixes which are not yet in MOR's DB and determines whether it is possible to find out the Direction (Country) for these new prefixes (Destinations).

This may be done in two ways:

1. The system checks the available prefixes for the best match and selects found Direction as a basis for the new prefix (Destination).

For example: if you try to import the rate for prefix 370630 and this prefix is unknown to MOR, MOR checks the longest matching prefix - 3706 in this case - and finds out that it is Lithuania Mobile. So it creates a new Destination with Prefix 370630 in which the Direction will be Lithuania.

2. If Country code is available in the imported file, MOR assigns this Direction to the newly created Destination.

For example: if again you try to import 370630 and it is unknown to MOR, but you have LTU in same CSV line for Country Code field, MOR will create such a Destination with Direction = Lithuania.

Rateimportfromcsv7.png

NOTE Because Destination cannot be empty (space), during the Destination creation, MOR automatically sets Destination name as Direction, Destination name will be Direction.

5.2. Updating Destinations

Step appears when Update Destination Names from file is check in second step.

6. Updating rates

To update existing rates your CSV file should contain prefixes that already exist in Tariff. Also, it should contain the exact Effective from date. If your CSV file does not contain Effective from date then you can set it manually in Step 2, just make sure that date is the same as in Tariff. Only then rates will be updated. If there are no existing rates with the exact prefix and effective from then new rates will be created.

Here MOR updates rates with the values from CSV files. These rates already exist in DB. Just click Proceed:

Rateimportfromcsv8.png

7. Creating new rates

Here, MOR creates rates that have not existed previously. Click Proceed afterwards.

Rateimportfromcsv9.png

8. Deleting rates

This is the final step. MOR deletes the rates which were not a part of the file being imported. You're all done!

NOTES

Rate import is VERY CPU-intensive.

If there are a lot of rates, please be patient and let MOR finish its job. It may take some time to count everything and come back to life.

At some point it may seem that browser is not responding. That is not the case – it is just working very hard, importing your very own rates from the CSV file.

If you get a timeout error, try to import the file in smaller pieces.

Destination to create from CSV

When importing rates from the CSV file, you can check which Destinations will be created automatically.

You can view them in a separate window when you click List in step 4 in the field "Destinations to create".

Such destinations do not exist in MOR's database but MOR will create them when importing the rate file.

More info: Integrity Check

Example Videos

<flashow>http://wiki.kolmisoft.com/videos/rates_import_from_csv.swf%7Cwidth=600%7Cheight=455</flashow>


Similar video with different tariff:

<flashow>http://wiki.kolmisoft.com/videos/tariff_import_from_csv.swf%7Cwidth=800%7Cheight=618</flashow>

Another example:

Another example with Provider tariff:

See also