Newswire Capture

Burli can ingest text wire data from XML files, XML streams, FTP sites or serial data streams in a variety of standards, languages and character sets.

Burli can also automatically ingest Audio feedsAudio files, RSS feeds, text (.txt) files, Podcasts, faxes and e-mails. Complete configuration details for each are available elsewhere on this website or from Burli Support.

To begin ingesting wire data, open Burli’s set-up dialog box (Shift-F8 on the keyboard or Config|Program set-up… from the menu), enter the configuration password, and select the Newswires-2 tab.

You may also see a Newswires-1 (or, in older versions simply Newswires) tab. This tab supports additional (generally older) wire formats, but unless you are specifically advised otherwise or already have wires configured on it, you can ignore this tab.

Installing a newswire

Find your newswire from the alphabetical list. Note that some newswires have different entries in the list for different regions (Europe/US), or delivery formats. Select the one that most closely matches the wire you wish to capture. 

If your wire does not appear in the list, please contact Burli’s Technical Support for more information. There may already be a generic wire (ANPA, NewsML) that you can use, or we may add specific support for your new wire.

Once you have selected your wire, press the Install… button.

For most wires a new screen opens with a series of options for how the wire data arrives. For some wires (notably FTP or XML wires) these options do not apply and only one set of configuration (usually File) settings appears.

Serial (COM port) newswires

Some agencies still provide serial data streams. If you are upgrading from an older newsroom system you may have one or more wires that still arrive this way and need to be moved to Burli.

Male end of a standard serial connector. Pin 2 carries the wire data and pin 5 is the ground.

In most cases the existing connection can simply be unplugged from the existing terminal and connected to a COM port on a Burli workstation. Burli conforms to standard pinouts for RS232 serial connectors: pin 2 receives data and pin 5 is the signal ground. Burli ignores the other pins.

 

On the COM Port configuration screen, select the port to which your wire is attached. Burli will accept any valid COM port that Windows can recognise, so multi-port serial cards that offer 8, 16 or more COM ports can be used.

Select the serial data settings. A few common presets are offered for each wire, but the settings can be edited manually by clicking in the Settings box and typing new values. If you are not sure about the settings your wire uses, contact the wire service directly for help.

Once you have entered the settings, press OK. On the newswires tab you should now see the settings beside the wire’s name. As Burli receives data, the byte count will begin to climb. Depending on the time of day and the wire, it may take several minutes for data to arrive. If the Received bytes never climbs above 0 it means no data is making into Burli. Check your hardware.

If the byte count rises but no stories appear in Burli (they will scroll along the Ticker Tape at the bottom of the screen and appear in the In-Queue on the main screen), then your COM settings are likely wrong. In this case Burli is receiving data but cannot read it well enough to determine where stories begin or end. Confirm that the Com port in Windows is configured the same way (baud rate, parity,stop bits) as it is in Burli.

TCP/IP Delivery

Some newer wires deliver their data over the Internet as a data stream, usually in XML format. This is NOT the same as XML file delivery (see below) which is far more common. Burli can connect to some TCPIP streaming servers, given the right connection details. However, security systems used by some agencies may prevent any software but their own from connecting to the servers.

On the TCPIP configuration screen, enter the IP address and port of the server you need to connect to. These details will be provided by the wire service. Some services also require an initialization string when connecting to the server. If so, enter the string in the space provided.

File delivery

This has rapidly become the most common form of wire delivery. The wire agency provides data files (usually XML, though sometimes plain text) which the newsroom system must ingest. The files usually arrive either on your LAN via the agency’s own software or over an FTP connection.

There are two methods of getting data files into Burli. One involves Burli simply collecting files from a local or network LAN location; the files have generally been dropped there by software provided by the wire service. The second method involves Burli collecting the files from an FTP server at the wire service.

UNC Folder

If you are ingesting data files from a network location (WAN, LAN or local hard drive) check the UNC Folder button and provide the path. Drive letters (C:\ or mapped drives such as V:\ are allowed). For UNC paths which include the machine names, be sure to begin with \\. You can navigate to the network location with the … (Browse) button at the right side of the Folder box.

Be sure to check that the Windows user that your capture workstation will normally log in with (you’re probably using an Administrator account during this installation, but that may not be who is logged in 3 months from now) has read and (if necessary) delete permission for the folder.

FTP Folder

If you are ingesting data directly from an FTP site provided by the wire service, then select the FTP folder option and enter the connection details. These will be provided for you by the wire service. To test the FTP connection (without downloading any data) press the TestFTP button beside the FTP folder setting. This will tell you whether the connection details you have entered will allow Burli to gather data from the site. If it fails you may need to check your firewall settings or the server connection settings. FTP generally uses port 21 which is sometimes bocked by some corporate firewalls.

More Settings

Both the UNC and FTP settings screens offer additional options, some of which you will need to set for successful data collection:

Include Sub Folders – will include any valid files it finds in sub-folders of the folder defined above.

Exclusive Access – will lock the file while Burli is ingesting it. Sometimes required for successful import of files over slow connections where multiple newsrooms may be using the same file. Not usually required (as multiple systems should be able to read the same file from the server at the same time without issue) and it may cause problems for other ingest systems if you turn it on unnecessarily.

Passive Mode – uses passive (PASV) mode to connect to the FTP server. Sometimes required to get through your (client-side) firewall.

Auto Purging Files – Deletes the file once Burli has successfully ingested it. Double-check that no other newsroom or system needs the same files before turning it on. Some services, however, will never delete the data files, so this setting may be critical to prevent folders from filling with old data. Some folders may have security rights that prevent Burli from deleting files, in which case this function will not work.

Files older than – Prevents Burli from ingesting files that have been on the server less than the specified number of minutes (values of less than 1 minute are not allowed). Over slow Internet connections it can sometimes take more than a minute or two for the complete data file to arrive in the folder. If this setting is not checked, Burli may try to ingest files that have only partly arrived. If you receive incomplete data, try using this setting. Please be aware that the time is relative to the FTP Server timezone. If the FTP site if not in the same timezone as you, do not check this box or contact Burli Support for additional settings.

Files newer than – Prevents Burli from ingesting files that are more than x hours old. Some folders contain data files that date back days or even months. When you connect to the server, you may want Burli to ignore the old files to save bandwidth and prevent the system from becoming cluttered with old data. Burli does keep track of which items it has ingested, so this setting is not needed in normal operation. But where you connect to a folder that Burli has not downloaded from in a long time, it can be very useful. Please be aware that the time is relative to the FTP Server timezone. If the FTP site if not in the same timezone as you, do not check this box or contact Burli Support for additional settings.

File Filters – specifies which file types Burli will ingest from the folder. Wildcards are allowed. Separate file types with a ; (semicolon) but no space. If this field is left blank, Burli will not ingest any files.
Examples:
*.* – will ingest any file it finds in the folder
*.xml – will ingest any file with a .xml extension
*.txt – will ingest any file with a .txt extension
*news*.xml – will ingest any file with news in the filename
*nml;*.xml – will ingest any file with either a .nml or .xml extension

Multiple feeds

A single Burli workstation with wire ingest licensed can ingest as many wire feeds as you like. Serial (COM) wire feeds are limited by the number of COM ports on a given machine; file imports are limited by network bandwidth to a given PC. Generally speaking it is a good idea to spread ingest out over several machines, both to distribute the workload and so that failure of a single PC does not mean you lose all of your wire services.

Uninstalling a wire

To stop capturing a wire service, open the Newswires tab in the configuration window, click ‘show installed newswires’ at the top of the screen, and select the newswire from the list that appears. Choose ‘Remove…’ and press OK when asked to confirm removal of the newswire. Burli will stop capturing the wire immediately.

Diagnostic Logging

If you are having trouble with any of your newswires or are trying to capture a newswire that does not appear in the list, it may be useful to write the raw wire data to disk as log files.

To create a diagnostic log of the raw data do the following:

  1. Install the wire (or a generic wire as directed by Burli’s Technical Support department) as above.
  2. Open the data tab in the Program Setup area.
  3. Click on the Event Logs… button in the centre of the screen.
  4. Place a checkmark beside the Newswires (1 or 2) log plus the Miscellaneous log.
  5. Burli will begin streaming the data to disk in the Logs sub-directory local Burli directory (usually C:\Burli\Logs\Newswires). Each wire logs to its own file.

The log data will be written until you un-check the Newswires log in the Program Setup area. The log file(s) can be read with any standard text editor. Burli’s Technical Support department may also ask you to send them the log files for further diagnosis.

© 2019 EUMEDIANET BV