Please find all SurveyCTO release notes below. We've bolded the most popular improvements.
If you're an existing customer, updates are free but generally not automatic. Login to the Manage Subscription page to see what version you're on; if you're not on the latest version and would like to update, just click the green Update now button.
If you're looking for our full documentation, please login first to your SurveyCTO account and then click for help in the upper-right.
Version 2.30 included the following improvements:
- Added the ability for form and dataset managers to organize forms and datasets however they wish: forms and datasets can now be ordered and grouped in whatever way makes the most sense (rather than always being listed in alphabetical order). Throughout the server console, forms and datasets are now organized together into the same sections, with the organization of those forms and datasets being set on the Design tab.
- Added support for a new "enterprise" version of SurveyCTO, for institutions that require finer-grained control over user roles and permissions, to facilitate the sharing of a single SurveyCTO subscription across multiple country offices or teams. (Contact email@example.com to inquire about enterprise subscription options.)
- Expanded data download options on the Export tab of the server console: now users can export encrypted forms directly from the website (if they have the private key), and choose between wide and long export formats.
- Added the ability to easily publish incoming form data directly to Zapier, from where that data can trigger innumerable outside actions or transit into innumerable outside systems.
- Added the ability to easily publish incoming form data directly to many other outside systems via webhooks.
- Added the ability to edit the field type for certain types of fields in the form designer: now you can change between select_one and select_multiple, decimal and integer, date and datetime, or calculate and calculate_here.
- Added a "pie" (vs. "bar") display option for select_one data in the Data Explorer, to show categorical data in a simple pie chart.
- Revised the percentages shown for select_multiple bar charts in the Data Explorer, to be based on the percentage of submissions that included each selection vs. the percentage of all selections.
- Added a basic search function into the Data Explorer, so that it's easier to find individual submissions.
- Added a new "case management" option to filter case lists by user role (access level) rather than user.
- Added the ability to export Stata .do files directly from the Design tab of the server console (just choose Download, then Stata .do template).
- Reworked the Stata .do file templates to accept either long- or wide-format exports, producing a .dta file structure that mirrors the .csv data structure.
- For forms that record information about speed-limit violations, added a friendly interface for warning about violations in the Data Explorer's submission-details view.
- Improved how dataset-form linkages are reported in the Your forms and datasets section of the Design tab (including an option to click for field mapping and other details).
It also included these smaller changes:
- Added an option to exclude note fields in the Data Explorer's submission-details view.
- Revised the username() function to clarify whether the currently-configured username has or hasn't been authenticated with the server. This prevents users from configuring one username offline, filling out a form, and then changing the username again when connecting to the server.
- Removed the special Case management section from the Collect tab of the server console. Now you can easily add or edit "cases" datasets in the Your forms and datasets section of the Design tab.
- Changed it so that users uploading revised form definitions for existing forms have to explicitly choose to upload a definition for the specific form they want to update (to cut down on accidental updates to other forms).
- Made it so no user can downgrade themselves by changing their own user role.
- Removed support for "desktop datasets," which very few users found valuable. (Support for "server datasets" remains. And desktop datasets will remain, for now, if somebody updating from an earlier version already has them configured.)
- Improved support for web forms to allow low-volume web requests (e.g., for small-scale data-entry forms or attempts to preview a new form draft) to be processed quickly – even when high-volume web requests (e.g., for larger-scale web surveys with lots of simultaneous users) are flooding in and causing traffic jams.
- Eliminated a possible pathway to losing work: if you work offline in the form designer, logout and login again in a separate browser tab, then try to save, the form designer will now allow you to resume without losing your unsaved changes.
- Made the form's public key (if any) read-only in the form designer: encryption keys should only be added as part of starting a new form, and they should never be changed.
- Made it so that changes in a user's access take effect immediately, even if the user is currently logged into an active session (as opposed to waiting for the next login).
- In SurveyCTO Sync, fixed a problem with exports of nested repeat groups that would sometimes omit repeat instances from exported .csv files.
- In SurveyCTO Collect, fixed an error that would appear when trying to display a field with an image but no text label ("Attempt to invoke interface method 'int java.lang.CharSequence.length()' on a null object reference").
- Corrected an error that had made it impossible to create a quality check to monitor for missing (empty) values.
- Changed the Go To Prompt screen to no longer trigger calculate_here calculations.
- When downloading old versions of form definitions from the Design tab, fixed the old versions to be ordered by version (rather than being in a random order).
- Fixed the summary of monthly usage and the indication of when monthly usage limits reset to be more accurate around the time of the actual monthly reset; also made it so the Refresh button refreshes the next-reset date and time.
- Made it so the server will refuse to accept empty (0-byte) submissions from devices; this way, if there is some device error (like a full SD card), the server doesn't end up with empty submissions that cause errors down-stream.
- If "Publish existing data" is not selected, fixed new Google publishing configurations to no longer publish one old submission before new submissions.
- Fixed publishing to Google Sheets to work for response values that begin with an = (and thus look like formulas); these will now have an apostrophe added to the beginning, so that Google doesn't interpret the value as a formula.
- Improved treatment of attached datasets when new versions of forms are uploaded, saved, and deployed: a new version of a form will automatically inherit the dataset attachments from the existing draft (if any), otherwise from the currently-deployed version (if any); and when deploying a draft, the now-deployed version will have whatever datasets were attached to the draft.
- Fixed the filenames of audio audits that start at a random time to properly reflect the time that they started.
- Made it so users filling out web forms don't need to have "cookies" enabled in their browsers to upload images or other file attachments.
- Ensured that read-only text fields will not be mistaken for note fields and dropped from exports or Stata datasets.
- Improved how the Data Explorer formats certain HTML labels (like headings, bulleted lists, and tables).
- Improved how the form designer shows HTML labels with single- and/or double-quotes.
- Fixed the "Next page" option in the online help to always start the next page at the top.
- After somebody purges data, made sure that the submission or dataset record count immediately updates in that person's server console.
- Improved the web display for field-list groups that include multiple note fields.
- In the Data Explorer's submission-details view, stopped trying to dynamically-substitute field values in labels: now, dynamic field references are simply left like .
- Corrected cases where, under heavy concurrent use of web forms, returning users might not be offered an option to resume a previously-abandoned web form.
- Improved the Data Explorer's handling of incomplete submissions that have been accepted (no longer offering missing attachments for download).
- Fixed the attachment-download function in the Data Explorer to work better in Safari.
- Fixed a problem that would prevent the Data Explorer from loading if a submission included an attached file with an apostrophe in its filename.
- Fixed the form designer's summary of fields to not be confused by single- or double-quote characters in labels that also have HTML code.
- Made it so the web and Android interfaces were consistent about handling HTML code inside field hints and dynamic field references.
- Fixed the form designer to not get confused when a media file is named such that its name is also within the name of another media file's name (like the name "image.jpg" is within "2-image.jpg").
- Fixed it so that pressing ESC to close a pop-up in the form designer won't leave you unable to scroll the main browser window.
- Improved the error message users get if they use a hyperlink from another user who was filling out a web form (since the other user's session information is in the URL).
- When looking up submission details by KEY, fixed the server to ignore cases where the same KEY is listed multiple times.
- Improved the error message for cases where somebody tries to attach a dataset to a form that has just been deleted.
- In SurveyCTO Collect, fixed text inside field references to be escaped, so that it won't be misinterpreted as HTML.
- Fixed an error about not being able to deploy a draft version of a form not to mistakenly use the term "sample."
- Fixed the public/private key generator to always add the proper filename suffixes, even when key names are really long.
- Removed the warning when downloading key files in Safari, since Apple has solved the problem that was breaking filenames for saved key files.
- Improved the error message when somebody uses the "look up by key" function to view a submission but then gives the wrong private key for decryption.
- Fixed links to other SurveyCTO servers inside web forms not to presume that the link is to another web form.
- Made it so that abandoned, unsaved web forms sessions are eventually deleted even for logged-in users (after 120 hours, vs. 48 hours for anonymous web forms sessions).
- Made it so the form designer only loads the first 1,000 columns and 50,000 rows of a form definition (in order to avoid spending too much time on worksheets that have thousands or millions of blank columns and rows).
Version 2.212 was a minor maintenance release to which all v2.21 users were automatically updated. It included the following fixes and small improvements:
- Improved the process by which new users are added to a SurveyCTO account: new users who login with email addresses will automatically receive an email notification about their new login, and administrators who add those users will be able to invite them to set their own passwords or include a pre-set password in the email notification.
- Added a more direct option for users to reset their passwords when having trouble logging in to their SurveyCTO servers.
- Added the option to add a sample form, complete with sample data, to the server. In the "Your forms" section of the Design tab, the Tools menu now has an option to add a sample household listing form with sample data that can then be explored or exported on the Export tab.
- Added the option to disable the Restore Data function from SurveyCTO Collect's Admin Settings, and changed the default for new devices so that Restore Data defaults to disabled. This can help to avoid cases where enumerators might accidentally restore old data.
- In SurveyCTO Collect, fixed a problem where some question screens might sometimes appear as blank on some devices, when HTML labels are enabled. This could occur randomly, for example, when swiping very fast through a lot of screens, and you might have to swipe away and then back again to get the questions to appear.
- On the server console's Monitor tab, fixed an error in the submissions chart caused by an April 2017 Google update.
- In SurveyCTO Sync, fixed a problem that occurred for some combinations of forms and data, where data in nested repeat groups might be omitted from exports.
- In SurveyCTO Collect, changed the default introductory screen, at the beginning of a form, to be centered even on devices with wide screens. (If there is a custom note with the "intro" appearance, though, then it will be left-aligned by default.)
- Got rid of an incorrect warning that would appear in quality-check results if quality checks were run very soon after creating the first quality check.
- Fixed a problem in the web forms interface, where a select_one or select_multiple field's current selection could become confused if there were negative and positive versions of the same choice value (with the negative value first). In such cases, selecting the negative value and then revisiting the field would result in the positive value being selected.
- Changed how web forms initialize so that servers can better handle a lot of users starting to fill out forms at the same time.
- Improved text auditing for fields within a single "field-list" group (i.e., within a single screen): all fields in the group will get the same timing information now, rather than no fields having anything recorded.
Version 2.211 was a very minor maintenance release to which all v2.21 users were automatically updated. It included the following fixes:
- Fixed the Data Explorer to properly render map-relationship views when they initialize off-screen.
- Fixed the Form Designer to work properly for forms with attachments that have apostrophes in their filenames.
Version 2.21 was a minor update. It included the following improvements:
- In the Data Explorer, added a new map relationship view, for when you relate a geopoint field with some other field; if the other field is numeric then the map pins will be shaded, and if the other field is a select_one then the map pins will be colored by value.
- In the Data Explorer, added a new trend view, for when you relate a date/time field with a numeric field; in this case, the numeric field will be graphed by time, allowing you to easily see time trends in your data.
- In the Data Explorer, added an option to show text-audit data in the submission-details view; now, with one click, you can see the time at which enumerators arrived at each field and the total time they spent on each.
- Added a "Duplicate form" option to the "Your forms" section of the server console's Design tab; this allows you to safely duplicate any existing form, resetting the title, ID, and encryption settings as appropriate.
- Added the ability to safely create new encryption keys right in the server console, from the global Tools menu at the very top of the "Your forms" section of the Design tab.
- Changed audio audits to record in .mp4 format, which is more widely-supported by web browsers and desktop computers.
- Added an option to directly hyperlink to view a submission's details; in the Data Explorer's submission-details view, there is now a button to copy a hyperlink directly to that submission, in order to share it with somebody (though a valid login is still required, plus the correct private key for encrypted forms).
- Added a "Look up by key" option to the "Your submissions" section of the server console's Monitor tab; this allows you to directly view submission details by key (or UUID).
- In the Data Explorer, improved the table view for date/time fields, so that you can control the level of aggregation (days, weeks, or months).
- In SurveyCTO Sync, improved Stata templates to have fewer naming conflicts when reshaping repeated data; now, Stata variable names for reshaped data will be named like "age__1" rather than just "age1" (using two underscores).
It also included these smaller fixes and improvements:
- In the Data Explorer, added a warning up-front to anybody with a web browser that is too old or otherwise incompatible.
- In the Data Explorer, improved the labeling of the x axis for date/time fields.
- In the Data Explorer, improved the formatting for HTML labels and multi-line responses.
- In the Data Explorer, made sure that the submission-detail view always scrolls back to the top when changing submissions.
- In the Data Explorer, improved the toolbars and menus for the submission-details view.
- In the server console, fixed the "Start new" wizard to ensure that form ID's don't include any illegal characters (like periods). Also fixed the Form Designer to prohibit illegal characters when editing a form ID.
- Changed the server to leave decimal values alone on export instead of trying to format them; this avoids "Null" errors in cases where field types have changed from something else to decimal (and some data in the field is non-numeric).
- Fixed the usage graph on the Monitor tab to no longer generate an error or disappear when Firefox or some browser add-on blocks Google Analytics.
- Made some small improvements to the server console interface for Microsoft Edge v14 users.
- In the Form Designer, added a "Normal" formatting option to the HTML text editor.
- In the Form Designer, improved the summary of a field label at the top of the edit screen, when the label has a wide HTML table inside it.
- In SurveyCTO Collect and SurveyCTO Sync, fixed local wi-fi sync to work for forms with UNICODE characters in their titles.
- In the Data Explorer, improved the performance of the floating yellow bar that indicates global filters or exclusions. Also made sure that the language-selector never opens behind this yellow bar.
- In the Data Explorer, made sure that table-view customizations are always saved.
- In the Data Explorer, fixed the submission-print view to include a title.
- In the Data Explorer, fixed a case where changing the title of a parent group would wrongly change the titles of child groups.
- In the server console, changed some "Cancel" buttons to "Close" to be more consistent.
- In the web forms interface, improved the sizing of HTML text in the "Go-to" UI.
- In SurveyCTO Sync, made the "replace linebreaks" option work for all exports, not just the long-format .csv exports.
- Throughout the web interface, stopped popping up error messages that come from browser plug-ins.
Version 2.20 focused on data visualization, exploration, and monitoring:
- Added the new SurveyCTO Data Explorer for flexibly and securely visualizing and exploring data – even when the data is encrypted using your own encryption keys.
- Removed all support for Statwing (it's been acquired and is no longer available for integration).
It also included a number of smaller fixes and improvements:
- Improved the "table" appearance for repeat groups, to have it take less than 50% of the SurveyCTO Collect screen when the current table is small enough.
- Fixed a problem with the "Go To Prompt" interface in SurveyCTO Collect, where some questions and labels would be jumbled when HTML labels were enabled.
- Corrected the "Add all" button when publishing to Google, to no longer skip every 10th field by accident.
- Improved performance for cases where many simultaneous users are filling out forms on the web.
- Fixed a problem where newer quality checks could cause older quality checks to no longer report new warnings.
- Fixed SurveyCTO Sync to no longer give security warnings on Mac OSX Sierra systems.
- Removed a vulnerability by which an error might cause some submissions to wait for later, newer submissions before publishing to Google.
- Corrected an intermittent problem with text audits when many simultaneous users are filling out forms on the web.
- Corrected an intermittent error with text audits when SurveyCTO Collect users on Android swipe through a survey form very quickly.
- Made it so you can use SurveyCTO Collect to submit data for a form even if the form's title has linebreaks in it.
- Limited the size of form attachments uploaded from Google Drive to the server console or form designer (100MB per attachment, 300MB overall).
- Fixed the form designer to properly load Excel form definitions even when they have zero fields.
- Fixed SurveyCTO Collect to not use Android M's fake MAC address as the unique deviceid.
Version 2.12 included a series of key improvements:
- Added support for rich HTML formatting in field labels, including a WYSIWYG editor built into the form designer. Now it's easy to format labels however you want!
- Added new "table" and "table-labeled" appearance options for repeat groups. With those appearances, users will see a table underneath each repeated field, allowing them to easily review and navigate data in the repeat group.
- Made a series of improvements to geopoint fields: improved GPS performance for many devices, added new support for showing and adjusting GPS positions on maps, added support for invisibly recording locations in the background, added a SurveyCTO Collect option to pre-warm GPS at the start of any form that collects GPS positions, and added an option to specify when to stop trying to record a more accurate location (the accuracy threshold, which defaults to 5m).
- Added help links into the form designer, so that it's easy to get to an appropriate help topic.
- Improved the SurveyCTO help system's search function. Now it's easier to find the right help topics!
- Made it easy to move existing fields and groups into a newly-created group in the form designer (so that you don't have to individually drag and drop each individual field or group into the new one).
- Added a Hindi interface to SurveyCTO Collect.
Other new features, improvements, and fixes included:
- Added an automatic reminder to download back-ups of forms before deleting them.
- Made it so SurveyCTO Collect will simply ignore blank values in the sortby column of a cases dataset (rather than giving an error).
- Added help for the time field type.
- Added a new option in the server console's Server preferences to stop update notifications from being sent to users of older versions of SurveyCTO Collect. Administrators who wish to keep using a custom-branded build of Collect (or an older version) can use this option to more easily prevent users from downloading the latest update.
- Fixed SurveyCTO Collect to properly show labels for multiple-choice selections in the Go To Prompt interface, even when the choices are dynamically-loaded and the submission has been saved and resumed.
- Made it so a user filling out a web form will have their current response saved when they click a hyperlink to jump to another part of the form.
- Fixed the round() function to round cleanly in cases that used to leave small trailing decimals.
- Made it so the constraint and calculation builders (in the form designer and elsewhere) will show calculate and calculate_here fields as options whenever a numeric field is needed.
- Changed the form designer to show groups with the "field-list" appearance with a different icon.
- Improved the experience of users who refresh the server console but have to re-login because their sessions have expired: no longer have multiple pop-up messages that need to be closed.
- Added a "please wait" message when loading large web forms.
- Fixed the mouse cursor in the main screen of the form designer, to always indicate whether you can or can't pick up a field or group to drag it somewhere else.
- Corrected a French label for the "Edit saved form" option in Admin settings to be French instead of Spanish.
- Got rid of the "GPS coordinates can only be collected when outside" hint when showing geopoint fields to users filling forms in a web browser.
- Fixed SurveyCTO Sync to properly export KML files when the geopoint field isn't selected to be included in the KML details.
- Added a check to make sure that new form versions that call pulldata() have the correct number of parameters.
- Added a message in the Start new form wizard to warn Microsoft Edge and Internet Explorer users that they might have to turn off their pop-up blocker to open the form designer.
- Fixed the calculation builder to properly order the terms when constructing subtraction expressions.
- Corrected the Download form files interface to properly indicate when a form has not yet been deployed.
- Fixed the Download to computer option for new forms to work even when the "Ripple Emulator" Chrome extension is installed.
- Improved handling of incomplete-submission detection in cases where forms are updated such that fields with file attachments change groups.
- Improved SurveyCTO Sync's "ignore groups" option to also ignore non-repeated groups that are inside repeat groups.
- Fixed the interface for uploading files from Google Drive to allow you to change your mind and upload from your local computer instead.
- Improved SurveyCTO Sync's handling of cases where a repeated image field is moved inside a nested non-repeating group.
- Fixed SurveyCTO Sync to properly show UNICODE form titles, even on Windows.
- Improved the error messages when people try to upload files that are too big (over 100MB).
- Improved performance when publishing large datasets to Google.
- Fixed it so that accidentally including "intro" or "thankyou" in a non-note field wouldn't cause the field to be invisible.
- Changed data exports done directly from the server (via the Export tab or via the API) to support up to a maximum of 10,000 columns. For exporting data with more than 10,000 columns, one can always use SurveyCTO Sync.
- Fixed the interface for publishing to Google to be more forgiving if somebody tries not to name the target Google Sheet or Fusion Table.
- Improved how the form designer shows very long relevance expressions.
- Fixed the Download printable version option not to let you click to download until the list of possible languages has been fully loaded.
- Fixed the configuration interface for publishing to Google Sheets to require that at least one non-repeated field be published.
- Revised the SurveyCTO logo everywhere, to match our subtly-updated logo.
Version 2.11 was a minor maintenance release. It included the following improvements:
- Added support for the new Community and Plus subscription plans.
- Began enforcing new subscription limits, and added a new interface for seeing (and being warned about) server usage.
- Made some improvements to the wizards used to build calculation and relevance expressions, to make them easier to use.
- Added a new option to display send/receive status on the SurveyCTO Collect main menu, to help increase transparency about connection status, background-sync settings, and more.
- Improved the form designer's performance when loading and working with very large forms.
- Added explicit support for proxy servers into SurveyCTO Sync.
- Fixed a problem that prevented web users from submitting forms when there was heavy simultaneous use combined with validation errors in the final attempt to submit.
- Updated the web UI to no longer offer to resume a previous submission if the case ID in the URL was different (so different cases are effectively treated as different forms now).
- Fixed a problem that could cause a web user's survey language to automatically change mid-survey.
- Fixed the web UI to use the latest Chrome and Firefox support for capturing images and other media (since browser updates made it so the camera would stay on even after capturing a photo or video, and the microphone would turn on even when it wasn't needed to capture a photo).
- Improved the documentation for quality checks (which are now supported in all SurveyCTO subscription plans).
- Fixed a small problem with daylight savings time sometimes causing times to be improperly shifted by one hour.
- Added support for longer email addresses (now up to 100 characters).
- Added a new "minimal appearance" mode to the web form UI, for embedding web forms into other pages with iframe tags (just add appearance=min to the URL query string).
- Changed the server to ignore columns beyond column 1,000 and to ignore rows beyond row 50,000 in uploaded form definitions (so that it doesn't waste huge amounts of time trying to process millions of blank cells).
- Fixed SurveyCTO Sync to properly exclude note fields from Stata .do file templates when configured to exclude notes from exports.
- Fixed it so the duration recorded for time users spent filling out forms would not be over-estimated because of repeated rounding errors.
- Removed Statwing integration from subscription plans that don't include it.
- Fixed constraint expressions generated by the built-in wizard to properly handle cases where punctuation or letters are disallowed.
- Fixed problem that prevented forms with periods (.'s) in their form ID's from being filled out on the web.
- Improved behavior of SurveyCTO Sync for forms with titles starting with periods (.'s).
Version 2.10 was a major release. Its most significant new features and improvements were:
- Added a new online, drag-and-drop form designer, to make building and editing forms dramatically easier. Form design in MS Excel, Google Sheets, and even XML is still supported, but the new designer is expected to become the primary form-design interface for new users.
- Added support for draft vs. deployed forms, so that revisions to a form can be safely reviewed before being deployed to live users.
- Made several improvements to the "Start new" process for starting a new form on the Forms tab (for example, to make it easy to start from a sample form rather than a blank form template).
- Removed all limits on the number of user accounts, for all SurveyCTO subscriptions. Now all team members can have their own SurveyCTO logins.
Other new features, improvements, and fixes included:
- Shifted to requiring email addresses for all SurveyCTO user accounts, except for data-collection-only accounts (which can now use either an email address or a non-email username). Users who try to login to the server console after upgrading will be asked for their email addresses and logged in under new accounts; old logins will be left in place in case any devices are configured to use them.
- Added new "single sign-on" support so that users could easily login from our public website, start using their server console, open the Manage Subscription page, and visit the Support Center, all without having to login more than once.
- Changed to using "value" instead of "name" for the column of the choices worksheet that indicates the internal value for each multiple-choice option.
- Added the new file field type for accepting arbitrary file attachments as part of a form.
- Extended the calculation builder to include additional calculation types (e.g., "current position in repeat group").
- Fixed the indexed-repeat() function to again work within choice_filter expressions.
- Fixed jr:choice-name() to again work on repeated fields that have choice_filter expressions, even when editing a saved form.
- Fixed a problem where quality-check reports would delete quality checks if they hadn't first been run manually.
- Improved the online help interface for users with smaller screens or browser windows.
- Fixed relevant-but-blank field values to overwrite prior values, if any, when merging data into a server dataset, Google Sheet, or Google Fusion Table.
- Fixed the Stata templates not to include an extra / (as in "C://") when saving to a Windows drive's root directory.
- Fixed video fields with the "new" appearance to work properly in web forms.
- Removed Enketo web form support (in favor of our built-in web forms).
Version 2.02 was a minor maintenance release. It included the following improvements:
- Extended SurveyCTO Collect's Edit Saved Form feature to allow users to resume from whatever position they had previously left off in a form. Also added the ability to configure which resume options should appear when editing a saved form.
- Added support for password expiration and minimum-complexity requirements. Users who wish to add password requirements for their SurveyCTO logins should contact customer support.
- Added support for users to review the past 30 days of server log files for their virtual servers. Users who wish to review their server log files should contact customer support.
- Extended SurveyCTO Collect's ability to be launched from other Android apps, to allow for more data to be passed from those apps into SurveyCTO forms.
- Extended the server's REST API to support JSON-formatted data responses, and to support partial (new) data downloads.
- Added Bengali translation for SurveyCTO Collect (thanks to Amplifier Strategies).
- Improved tracking of durations (via the duration() function) for users filling out forms with the web interface: now, time spent between long absences will be included in the logged duration.
- Improved the auto-generated Stata templates so that they would automatically detect regional settings where dates are not in month-day-year order and adjust the generated Stata code accordingly.
- Fixed a few cases where incomplete submissions that are manually marked as complete might either be left out of server exports or cause duplicate records in server exports.
- Fixed a problem where selecting the SurveyCTO Sync set to include group names in .csv headers option when generating a mail-merge template would generate an error.
- Changed data exports done directly from the server (via the Export tab or via the API) to give up at 100,000 columns. For exporting data with more than 100,000 columns, one can always use SurveyCTO Sync.
- Automatically stripped any unprintable or illegal characters from scanned barcodes in SurveyCTO Collect (in order to prevent errors saving submission data).
Version 2.01 was a minor maintenance release, to which all 2.00 users were auto-updated. It included the following changes:
- Fixed it so that expressions for choice filters (a.k.a. "cascading selects") can reference repeated fields (so long as the referenced fields are within the same repeat context as the expressions).
- Improved compatibility with Microsoft Internet Explorer and Edge browsers.
- Improved performance for servers with hundreds of user accounts configured.
- Improved performance for web users finalizing forms.
- Changed the form preview feature to no longer offer to resume previous sessions; now every time you preview a form, it starts fresh using the latest version of that form.
- Improved compatibility for anonymous web users, so that they can fill out forms without enabling "cookie" support in their browsers.
- Added support for the name column of the choices worksheet to be called value instead. (In a future release, we plan to use this more intuitive column name in samples and form templates.)
- Improved the error messages for users who try to login to SurveyCTO without "cookie" support enabled in their browsers.
- Extended server to auto-correct names of incoming text audit files when necessary. This avoids submissions being marked as incomplete for some cases where there were device-side file system errors.
- Fixed quality checks to always consider all submissions, even ones with all-blank values.
- Fixed errors that would result from previewing a form in multiple browser windows, if one or more datasets were attached to the form.
- Fixed a problem where errors could result from configuring a dataset to use a certain field as the unique ID, if that field already contained data with duplicate values.
- Fixed SurveyCTO Collect not to stop with an error when a form calls index() multiple times.
- Fixed printable surveys, form previews, and web forms to properly display linebreaks inside hints and group labels.
- Added the ability for users to review server logs (including all logins and other server activity).
- Fixed a very occasional problem with how some browsers displayed radio buttons in web forms.
- Fixed the server to allow you to switch from uploading from your Google Drive to uploading from your local computer, without having to refresh the page.
- Fixed the rank-value() function to work properly in the case of ties.
- Fixed the server console's editor in the case management section, to no longer create duplicate cases when the same row is edited multiple times.
- Fixed a problem with the REST API whereby certain sequences of queries could result in some data being duplicated in the returned .csv files.
- Improved the server console alert that appears when one's number of allowed web submissions has been reached for the month.
- Added a warning to discourage users from configuring SurveyCTO Sync to use a Dropbox or Boxcryptor folder for local storage.
- Added a warning on form upload for cases where fields by the same name appear in multiple groups.
- Fixed situation where, after manually deleting the cases dataset, the case management section of the server console would still report that there were cases.
- Fixed SurveyCTO Sync to properly download preloaded data files when downloading form definitions via local wi-fi sync.
- Fixed an exceptionally-rare problem where a crashed form would resume with a new version and then, after being encrypted, get errors on decryption.
- Improved the behavior of server console tabs when you switch between them (new tabs start at the top, previously-viewed tabs revert to their prior positions).
- Improved some panels on the server console to auto-scroll into view, when they open mostly or completely off-screen.
- Fixed a few error messages that had broken links to online-help topics.
- Fixed the date-time() function example in the help to have the proper syntax.
- Made it so that you could resize all windows in SurveyCTO Sync, including the license-verification window.
- Made it so the contact support option in the help section goes directly to the page for opening a new support case.
- Fixed an error that could result from attempts to publish encrypted data from pre-v1.40 to a dataset.
- Fixed a rare bug where clicking the device's back button while a form was loading could cause SurveyCTO Collect to crash.
- Fixed a problem with the graph of submissions on the Monitor tab for new users.
- Improved the degree to which elements of the server console's Design tab update immediately when you upload a new or revised form.
- Fixed the server console to give a comprehensible error when you try to delete a form that is already in the process of being deleted.
- Added a note to the help topic on using Enketo, to mention that we'll be discontinuing support for Enketo in a future release.
- Improved server performance for forms with many thousands of submissions and attachments.
- Started limiting datasets to 1,890 fields (rather than 4,000) in order to avoid potential errors when submissions include large amounts of text. Also improved error message for the case where a form publishing to a dataset tries to publish too many columns, and fixed case where an uploaded .csv file for a dataset violates the limit.
- Fixed a rare server console login error that would happen if you tried to upload a form using a browser window with an expired session.
- Fixed the server console to require that a field selected as the unique ID for a dataset also be published to that dataset.
- Updated the online help for local wi-fi sync to advise on the necessary firewall configuration.
- Made small improvements to the online help, including the navigation.
Version 2.00 was a major release. Its most significant new features and improvements were:
- An entirely new server console. This console features a more modern, easier-to-use interface that better organizes SurveyCTO's growing range of capabilities and online help resources.
- A web interface for previewing and filling out forms. This allows users to easily preview forms for testing, fill out forms in-office (e.g., for a telephone survey), and share forms for others to fill out online.
- Direct access to data, analysis, and visualization. The new server console's Export tab allows you to download data – or even start to visualize or analyze it – with a few simple clicks. For visualization and analysis, SurveyCTO has integrated directly with Statwing's powerful cloud platform (and SurveyCTO automatically covers all of their fees for professional-level subscribers).
- Direct publishing of form data to Google. Rather than having to publish form data to a dataset in order to then publish that dataset to Google, you can now publish forms directly to Google.
- Direct configuration of quality checks for form data. Rather than having to publish form data to a dataset in order to then configure quality checks for that dataset, you can now directly configure automated quality checks for forms.
Other new features, improvements, and fixes included:
- Renamed SurveyCTO Client to SurveyCTO Sync.
- Added new "intro" and "thankyou" appearance styles for note fields, so that forms can include custom content on the opening and closing screens of a form.
- Extended support for exporting dummy variables (1/0 columns) for select_multiple fields, to cover more export formats and cases.
- Added an option in SurveyCTO Sync to exclude note fields from exports (since they never contain any actual data).
- Added an option in SurveyCTO Sync to merge old and new data in cases where fields have changed groups over time (this is the new "ignore groups" option).
- Improved the performance of server datasets, so that data can stream into them more quickly (for projects with many enumerators sending in data simultaneously).
- Fixed the constraint tester to properly show the field name when selecting an existing field that happens to be inside a group.
- Made several performance improvements in SurveyCTO Sync: starting to listen for submissions on a local wi-fi network should be quicker, and systems with datasets downloaded to the local storage should face fewer delays.
- Added support for resuming failed downloads of SurveyCTO Sync or SurveyCTO Collect.
- Fixed the server to be able to properly generate printable versions of forms that include calculated repeat_count expressions and calls to the username() and caseid() functions.
- Fixed it so that incomplete submissions that are later completed do not result in duplicate server dataset records (they will only publish to the dataset once, when they are completed).
- Made SurveyCTO Collect more tolerant of cases where the underlying Android directory structure changes (which could happen when upgrading the device or moving data from one device to another).
- Added support for a new file field type for arbitrary file uploads – but haven't documented it yet since it currently works only in the web interface.
- Improved the format of data sent by SurveyCTO Sync to Statwing, so that visualization and analysis on the Statwing side is easier (e.g., select_multiple fields are sent in a certain format that allows Statwing to group responses together as checkboxes).
- Added a new hash() function to allow generation of hash values for passed parameters (can be used by advanced users to obscure/encrypt data that is published, e.g., as pre-loaded data).
- Fixed desktop datasets to support exported values over 255 characters in length.
- Shifted from recommending the old position(..) function for getting the current repeat-group position to recommending instead the new index() function. This new function will give the index number you want even when called within a non-repeated group nested within the repeat group.
- Improved the error messages given in cases where there are problems encrypting or decrypting form data (often it's just that the person trying to decrypt data is using the wrong private key).
- Fixed SurveyCTO Sync to be able to properly export desktop datasets that have names starting with numbers.
- Fixed SurveyCTO Sync to export 1/0 columns for missing select_multiple responses as missing, rather than as all 0's.
- Fixed server datasets to enforce a reasonable limit on the overall number of fields rather than crashing when the number of fields becomes absurd: server datasets will now be limited to a maximum of 2,000 fields.
- Extended the REST API to support encrypted forms: if somebody requests an encrypted form's data via the API, the server will export and return a .csv file that includes all publishable fields.
- Fixed server datasets not to get confused by fields that have names that look like repeated-field names (like age_2).
- Added an option to set a SurveyCTO server to require all forms to be encrypted. Contact support if you'd like to request that your server be set to this "only encrypted" mode to help enforce your data-security policies.
- Fixed the "month-year" appearance to work again for date and datetime fields shown on devices running Android's "Lollipop" release.
- Added a few example fields into the default form template, to give new users a bit more to work with.
- Added the option to clean old datasets from local storage in SurveyCTO Sync.
- Improved the error message for cases where pre-loaded data includes empty values.
- Fixed documentation for the substring() function to be more correct.
- Fixed Google publishing to better support target column names that begin with numbers.
- Heavily revised (improved!) online help content.
Version 1.402 was primarily a maintenance release. It included the following improvements and fixes:
- Added a new duration() function that returns the total amount of time spent, in seconds, filling or editing the current form submission. Call this function in a calculate_here field to capture the user's number of seconds into a form when they reach a particular point (expression for the calculation column: "once(duration())"); you can then subtract one captured duration from another to get the time spent in between. Or, use in a regular calculate field to capture the total duration spent on the form overall (expression for the calculation column: "duration()").
- Improved text and audio audits so that they now resume after saving, exiting, and then editing a form. If a certain audio audit is configured to begin 5 minutes into a survey, for example, but the user saves and exits after 2 minutes, then the audit will begin 3 minutes after the user resumes editing the form. Likewise, text audits will now continue to log for all editing sessions.
- Made it so data published to Google Spreadsheets doesn't always have to publish to the first worksheet: if there is a worksheet named "data", SurveyCTO will publish there even if it's not the first one. This allows for the first worksheet to be a dashboard or other worksheet, with the raw data appearing on a subsequent sheet.
- Extended SurveyCTO Collect to automatically unzip all files in .zip files attached to forms, regardless of whether those files were included in subdirectories (Collect used to only unzip the contents of the root directory, and it would even miss some of those files if subdirectories were present).
- Fixed a problem in Collect where you couldn't move forward if there were multiple calculate_here fields in a row.
- Fixed a one-in-a-million vulnerability that would cause serious database slow-downs when many users send submissions for the same form at the same time.
- Fixed it so that select_one fields with cascading-select filters and spaces in their choice values work properly when editing saved forms (previous selections were being lost when saved forms were re-opened).
- Fixed it so that you could download the printable version of a form even if that form used the pulldata() function in a repeat_count column and even if that form had repeat groups that might repeat forever (or a very large number of times).
- Fixed the server login screen to give an appropriate error about needing to be refreshed if cookies are cleared (rather than confusingly claiming that the username or password is incorrect).
- Fixed audio audits not to get confused about when to start when returning from recording audio or video (audits scheduled for later in the survey would start immediately upon returning).
- Fixed publishing to Google Fusion Tables to correctly publish data containing backslashes (rather than giving an error).
- Fixed Google publishing to fully cancel any ongoing publishing operations when a dataset is manually cleared of all data while in the middle of publishing.
- Fixed client datasets so that multiple fields with the same name could be safely merged from different forms into the same target column.
- Fixed SurveyCTO Collect to allow you to delete blank forms that have duplicate definitions (a rare event for people with multiple servers) even when there are unfinalized submissions pending.
- Fixed Stata .do file templates to always format the incoming SubmissionDate column as a Stata-format date/time field.
- Fixed Stata .do file templates to always include the full file path when calling out to other .do files (for processing nested repeat groups).
- Fixed it so that those with standard subscriptions over their user limit can delete users to get back under the limit.
Version 1.401 was a minor maintenance release. It included the following changes:
- Added ability to bundle specific settings, forms, and cases into custom-branded builds of SurveyCTO Collect (available for an additional fee for those who want a more customized and plug-and-play experience for their users).
- Further improved the performance of the SurveyCTO Client so that exports will be faster for most users.
- Added a new appearance for image, audio, and video fields to require the captured media to be new: when the "new" appearance is specified, the user will not be allowed to choose an existing image, audio, or video file on the device.
- Extended SurveyCTO Collect's setting for the number of days to back up finalized submissions to allow up to 90 days, and added a graceful error when internal storage space becomes too low to back up new submissions.
- Fixed publishing and wide-format exports of repeated fields to include fields inside non-repeating groups nested within repeating groups.
- Fixed publishing of repeated fields to work properly for cases where the repeated fields were encrypted but marked as publishable.
- Fixed fields with calculated default values to allow users to delete the pre-filled values (without re-adding the calculated default values back again).
- Fixed the server to allow users to manually accept incomplete submissions (those with missing attachments).
- Fixed the server not to claim attachments were missing or cause download errors when really they were all there (but some of them had been sent after the form was updated).
- Fixed repeated select_multiple fields to properly export into wide-format export files as series of 1/0 columns (when configured to do so).
- Fixed not-relevant select_multiple fields to properly export as series of 1/0 columns (when configured to do so).
- Fixed calculate_here fields at the very end of repeat groups not to cause errors in SurveyCTO Collect.
- Fixed dataset .csv file exports to enclose linebreaks in quotes so as not to confuse Excel or the SurveyCTO Client.
- Fixed scheduled syncs to work again in the SurveyCTO Client.
- Made it so the server login screen will give a more clear error when there is a browser plug-in interfering with server communication.
- Improved client datasets to better handle repeated fields (both in the choose-fields interface and in exports).
- Fixed the client not to complain when exporting client datasets that happen to be empty.
- Fixed form titles with illegal filename characters (like colons) to work more reliably throughout the system.
- Fixed SurveyCTO Collect not to crash when attempting to resume a 0-byte submission (which can be caused by an out-of-disk-space error).
- Fixed the server to accept form uploads when the form definition file is included both as the definition and as an attachment.
- Fixed the printable version of forms not to give errors when encountering hand-coded XPath expressions.
Version 1.40 focused on the removal of limitations:
- Removed the restriction on updating existing survey forms in ways that affect the data structure. Nearly all updates to an existing form are now allowed, including updates that add fields, remove fields, or otherwise change the structure of collected data; the only restriction that remains is that you can't change an existing form's encryption status or settings: you must change the form's unique form ID if you want to alter its data encryption.
- Removed the 81-field restriction on datasets so that over 1,000 fields can now be included in a single dataset (and, e.g., published on to Google).
- Expanded datasets to support fields in repeat groups, so that repeated data can be published into a dataset (and then, if desired, on to Google). Repeated fields publish in "wide" format, as when a repeated name field publishes into name_1, name_2, etc. columns.
- Added a new option to immediately publish all existing data when linking a form to a dataset.
It also included the following smaller improvements and fixes:
- Increased the speed with which the SurveyCTO Client exports data (for some forms on some computers, the improvement is massive).
- Added a new option to the SurveyCTO Client's preferences, to output missing values as something other than blank/empty. This allows one to distinguish, in exported data, between blank or empty values entered by the user vs. those fields that did not appear for the user: for fields that did not appear (either because they were not relevant or because they were not present in the version of the form filled out), any specified character or string of characters can be used to signify a missing value.
- Added a new formdef_version column to all exported .csv files. For each row (i.e., for each submission), that column will contain the version number of the form definition used on the device.
- Added a new option to export the internal KEY, SubmissionDate, and formdef_version columns to Excel, Google Earth, and other formats supported by the SurveyCTO Client; also added these internal columns to the list of fields that can be published to datasets.
- Added a new option to attach media or .csv files to forms validated offline (when installing them into local client storage for wi-fi sync).
- Added a new facility to clean out old forms and data from the SurveyCTO Client's local storage.
- Made it so SurveyCTO Collect will refuse to delete a blank form if there are unfinalized submissions begun with that form. This will prevent some users from getting into trouble.
- Fixed SurveyCTO Collect to be able to enforce "speed limits" even when no form fields are configured to record attempted violations.
- Fixed it so that local wi-fi sync could be used even for an encrypted form with an embedded server URL.
- Fixed the SurveyCTO Client to download and export datasets even if there was an error downloading forms.
- Updated Stata templates to drop all auto-generated label fields for groups with the table-list appearance.
- Fixed an export problem that existed for repeated GPS fields when those fields were located at the very beginning of a repeat group.
- Fixed a problem with mail merge templates where group names were being assumed part of column names for GPS fields, even when group names were turned off in the SurveyCTO Client preferences.
- Fixed publishing to Google Spreadsheets to allow columns with names beginning with numbers.
- Fixed publishing to Google Spreadsheets to survive Google's deprecation of one of their API's (as of April 20, 2015).
- When re-encoding .csv files with the SurveyCTO Client, updated to show only the first 50 rows of the .csv file in the preview window (so that the preview will be quicker for large .csv files).
- Improved some warning and error messages (as always).
Version 1.31 was the culmination of a series of small post-1.3 updates. Beyond what was included in earlier maintenance releases, it included:
- Dramatically improved ability to operate without an Internet connection, using local wi-fi networks. This replaces the earlier option of directly connecting devices via USB (which was, frankly, too tedious and error-prone).
- Changed Excel exports from .xls to .xlsx format and increased the maximum number of exported columns from 256 to over 10,000.
- Added a REST API for programmatic access to .csv exports directly from the server. (This isn't documented in the online help, so contact support if you are interested in using this API.)
- Added a new option to the SurveyCTO Client preferences, to replace all linebreaks in .csv exports with some other character. This defaults now to changing linebreaks to spaces because Excel too often messes up when importing cell values with linebreaks in them.
- Added a new help topic on how to import .csv files into Excel (since Excel has so many quirks).
It also included the following fixes and smaller improvements:
- Fixed Google Spreadsheets publishing to properly merge on numeric and mixed text-and-numeric columns, regardless of the cell formatting on the Google side.
- Improved server and Collect behavior when a repeat group is accidentally designed to keep repeating forever.
- Made it so none of the automated quality checks will fail when there's no data yet to check.
- Fixed it so that Collect's Admin Settings would properly remember the "Skip to Next" setting when different from the "Go to UI" setting.
- Added support for audio audits to begin or end at fields within field-list groups.
- Improved some warning and error messages (as always).
Version 1.302 was primarily a maintenance release. But it included the following significant changes:
- Markedly improved performance for SurveyCTO Collect, particularly when loading, navigating, and validating long, complex forms. (This was the result of a particularly rewarding collaboration with the core ODK team. Thank you, Mitch!)
- Added a new SurveyCTO Client option to export repeat groups in "wide" format. (Additional (title)_WIDE.csv files will be exported for every form that includes one or more repeat groups; these files will include a variable number of extra columns to hold data for all repeated fields.)
- Added a new SurveyCTO Client option to export select_multiple data as a series of 1/0 columns (i.e., as a series of "dummy variables").
- Updated the SurveyCTO Client to allow inclusion of repeat-group data in Excel, Google Earth, StatWing, and .csv-subset exports.
- Updated the SurveyCTO Client to respect the "include group names in column headers" option in Excel, Google Earth, StatWing, and .csv-subset exports.
- Made it so that SurveyCTO Collect will notify users when a newer version is available on the server (and link to the download via the Admin Settings screen).
- Added a new sample form that demonstrates five methods of asking follow-up questions based on user selections (e.g., asking follow-up questions for selected crops or acquired assets).
- Added a new help topic for migrating ODK users, to help quickly orient them.
It also included the following fixes and smaller improvements:
- Fixed a problem where you couldn't publish server datasets to Google Spreadsheet columns if those columns had spaces or punctuation in their headers.
- Corrected some rare and as-yet-unreported problems with calculations in complex forms (particularly in those involving repeat groups).
- Added a linebreak() function for cases where you want to include a linebreak in a calculated expression.
- Updated the rank() function to make it more useful within repeat groups.
- Relaxed a technical server limit on the number of possible fields and groups in a form.
- Fixed Collect to save audio and text audit files in more cases (e.g., even in the case of some crashes, and even if they are not-relevant during one or more mid-form saves – as long as they are relevant at the end).
- Made it so that you can once again accept incomplete encrypted forms (encrypted forms with missing attachments) if and when necessary.
Version 1.301 was a minor maintenance release. It included the following changes:
- Streamlined the user experience for publishing lots of form fields to a dataset; you can now "select all," and the dataset's field name will default to the form's field name so that you don't need to re-enter anything.
- Improved the first-time user experience for the SurveyCTO Client: now users won't need to install the Java cryptography update unless and until they need to work with encrypted forms.
- Restored the friendly error message that appears when trying to scan a barcode without the barcode-scanner app installed.
- Fixed a problem where media fields (for recording audio or video) wouldn't work unless there were audio audit fields configured.
- Corrected an issue with saving audio and text audits on some devices with non-Western locales; media filenames and text audits will now always use Latin numerals.
- Added a "please wait" pop-up for when a user filling out a form clicks the validate-now button. For very long forms that take a while to validate, it was appearing to do nothing.
- Made it so that you can switch back and forth between the 1.3x and 1.2x versions of Collect without running into database versioning errors.
- Fixed printable versions of forms to support groups with no visible fields.
- Fixed a problem with the Download as PNG and Download as PDF buttons on the Statistics tab.
- Corrected a rare issue where the Statistics tab would get stuck refreshing (and require a browser restart).
- Made it so that you can't accept incomplete encrypted forms (encrypted forms with missing attachments) because they can't be decrypted when incomplete.
Version 1.30 included the following new features:
- New support for monitoring the quality of incoming data. This includes checking for outliers, out-of-bounds values, field means that are too high or too low, field values that are too frequent or too infrequent, and even means or categorical response distributions that differ by group (e.g., when there are interviewer effects). See our page on collecting quality data to learn more...
- New support for enumerator "speed limits." These allow you to configure the minimum time that should be spent on particular survey fields; you can quietly record details about violations, automatically trigger audio recordings to help you investigate violations, or simply enforce the speed limits.
- New support for case management. Allows you to center your data-collection workflow not on forms but instead on the houses, schools, health centers, patients, or other subjects of your data-collection efforts.
It also included the following smaller improvements and fixes:
- Added a lengthy new help topic on assuring data quality.
- Dramatically improved the performance of the server interface for publishing datasets to Google Spreadsheets, for cases where users have a large number of Google Spreadsheets.
- Updated the client's UI to make it easy to sync multiple times without having to restart.
- Added a help topic to describe what to do if Excel gets really sluggish when editing a large form.
- Fixed occasional errors that could occur when manually deleting repeat-group instances (using a long-press followed by Delete Group in Collect).
- Changed all download links for the client and Collect to be un-encrypted, so that those downloads will be faster and more trouble-free.
- Fixed the server to accept form definitions pushed by ODK Briefcase, to ease ODK-SurveyCTO transitions.
- Added support for as-yet-unreleased versions of Java, so that future Oracle releases don't cause problems.
- Fixed the version() function to always return the form version, even for cached forms.
- Added a warning to Collect if running from the device's SD card (since performance and stability would be improved by moving the app itself to internal storage).
Version 1.231 included the following new features:
- Added support for publishing to Google Spreadsheets, for professional-level subscribers. This allows for automated streaming of data into Google Drive for real-time monitoring, publishing, and more. See our page on datasets to learn more...
- Improved options for validating, saving, and finalizing forms, as well as a lengthy new help topic that discusses the various options and workflows.
- Made use of pre-loaded data easier by including an option to automatically include the necessary fields when downloading a form template.
It also included the following fixes and smaller improvements:
- Added the option to open online help into a separate window, so that it can be easily printed or used side-by-side with the server console.
- Changed it so that form definitions and attached .csv files are always compressed in transit, so that form downloads will be quicker over slow connections.
- Extended pulldata() to support geopoint fields, so that you can both publish and pre-load GPS locations.
- Added the new rank() function for determining the rank order of repeated fields.
- Improved the performance of hyperlinked field references, to reduce delays when swiping from one question to the next.
- Added a new help topic on improving the performance of long, complex forms.
- Updated Android home-page shortcuts to survey forms to automatically open the latest available version of a form.
- Improved the catching and handling of errors relating to publishing of datasets and integration with Google Fusion Tables (column names with spaces, over-long column names, over-long column values, too many fields).
- Fixed date-time fields (like survey start and end times) to properly adjust the date when time-zone changes shift the time past midnight. This corrects some cases where incorrect dates would result from time-zone differences between the data-collection device and the data-exporting computer.
- Made the repeat_count column in the survey definition more forgiving about string values vs. integer values. Whenever it can, it now tries to automatically convert strings to numbers, rather than reporting errors.
- Made the selected() function more forgiving about string values vs. integer values. Whenever it can, it now tries to automatically convert numbers to strings, rather than reporting errors.
- Fixed the process that generates Stata templates to allow for cases where multiple-choice options do not have labels (but only pictures).
- Made it so Collect plays a notification sound when finished downloading forms or uploading submissions (if the device isn't muted).
- Added the ability to increase the length of a text field beyond 255 characters (though it's still undocumented, so contact us if you need this capability).
Version 1.23 included the following new features:
- Improved support for form navigation, including the ability to add hyperlinks to field groups, color groups and fields based on whether or not they've been completed, and organize the overall list of survey fields by group.
- Added support for publishing to Google Fusion Tables, for professional-level subscribers. This allows for automated streaming of data into a simple, free system for data visualization, for real-time monitoring, publishing, and more. See our page on datasets to learn more...
It also included the following fixes for rare problems:
- Fixed the form upload/conversion process not to take forever when groups are nested very deeply.
- Fixed the manual .csv upload process for datasets to no longer drop 1% of imported rows for .csv files over 100 rows in length.
Version 1.222 was a minor release. It included the following new features:
- Added support for pushing new survey forms to devices via USB, for professional-level subscribers. Combined with the offline sync capability added in v1.22, this allows for fully-offline operation in settings that require it.
- Improved handling of extremely long calculation, relevance, or constraint expressions. If loading a form in Collect more quickly on the second or subsequent load would result in a "stack overflow" crash due to an expression longer than the device can handle, Collect will now automatically fall back to a slower method of loading the form. This should allow the device to handle the expression in most cases.
- Added support for "geoshape" and "geotrace" field types, to support new features in Enketo Smart Paper.
It also included the following fixes and smaller improvements:
- Added desktop client support for Java 8, the newest Java platform released by Oracle this year.
- Improved desktop client support for Windows 8.1 (allowing more windows to be resized in case the default font and window sizes truncate window content).
- Renamed the enclosed-area() function to just area(), in order to harmonize with the broader ODK community (while maintaining backward-compatibility with any forms that use the old enclosed-area() name instead).
- Stopped opening Enketo Smart Paper pages as pop-up windows, in order to avoid issues with pop-up blockers.
- Fixed the root cause of very occasional "more than one fileset" errors: made it so canceling a form upload mid-processing and re-starting that same upload would not sometimes create duplicate database records on the server.
- Made some improvements to error messages (as always).
Version 1.22 primarily added:
- Support for powerful new workflows using datasets. See our page on datasets to learn more...
- Support for pulling data directly off of devices.
v1.22 also introduced a new tiered subscription model wherein not all SurveyCTO users have access to unlimited use of all SurveyCTO features; some SurveyCTO users now pay a discounted rate in exchange for some limitations. Existing paying customers were "grandfathered" in, so that they could continue to pay their existing subscription fees and yet retain unlimited usage – but these customers do not automatically gain access to all new features.
In addition, v1.22 included the following other improvements:
- Added the ability for all subscribers to use server datasets to update pre-loaded form data without having to update the form itself. See our page on datasets to learn more...
- Added the "show_formatted" appearance style for integer and decimal fields, so that users can more easily avoid mistakes in long numbers. For example, depending on the Android device's locale settings, "120000" may appear as "120,000".
- Added the "numbers_decimal" and "numbers_phone" appearance styles for text fields, so that different variations of the numeric keypad are available.
- Added the format-number() function to format a number according to the Android device's locale settings (so, e.g., "120000" may appear as "120,000").
- Added support for Unicode characters in mail-merged data.
- Changed calculate_here fields so that they do not calculate when the user views nearby fields in the Go To Prompt interface, but only when the user swipes past them in the course of filling out the survey. This improves the accuracy of "module start times" when calculate_here fields are used to calculate those start times.
- Improved some error messages (as always).
- Broadened support for pre-loaded .csv files to support Unicode files that start with a BOM character.
- Fixed Collect to properly display long decimal fields for editing.
- Changed Collect to default to 14 days of internal-storage backups rather than just 2.
- Updated the client's form validation to better catch and report all errors that would be caught and reported on upload to the server.
- Fixed a problem where calculation and relevance expressions were sometimes not evaluated in a proper sequence for groups that became relevant mid-way through a survey.
- Included ODK fix for Nexus 7 video capture on Android 4.3.
- Fixed a problem where single-word constraint messages were not being displayed.
- Added documentation for the new ODK pow() function to raise fields, numbers, or expressions to a given power.
Version 1.212 was a maintenance release that fixed just a single issue:
- SurveyCTO Collect defaulted to maintaining 2 days of back-ups in internal device storage. In prior releases, setting the days to auto-back-up above 25 would result in no back-ups being stored. This has been fixed so that you can save more than 25 days of back-ups.
Version 1.211 was a very minor maintenance release. It included the following small changes:
- Added the de-duplicate() function to remove duplicates from a string-separated list of values (useful when generating a combined list from repeated multiple-choice selections).
- Added support for an optional "style" column on the settings worksheet of form definitions, to support a new feature in Enketo Smart Paper.
- Fixed the printable versions of survey forms to include visible fields that have calculated defaults.
- Updated exported .kml files so that they can be loaded into Google Maps (in addition to Google Earth).
- Corrected inconsistent behavior and a potential crash in Collect when clicking Go To Prompt, then canceling before a very long form's list of fields has been rendered.
- Updated the client so that it consumes less memory when run for long periods in the background.
- Fixed Collect to be able to restore a crashed form even if its instance name was empty (a potential issue only for those dynamically constructing instance names).
- Improved the error message for cases where invalid column names are configured for a dynamically-loaded multiple-choice field.
- Resolved a case where Collect could crash when trying to use the Go To Prompt interface to navigate to a not-yet-created repeat group.
- Fixed cases where the client could show unnecessary errors when editing options for .kml exports or options for data-subset exports.
Version 1.21 was primarily a maintenance release, but it did include three new features:
- Added the distance-between() function to calculate the distance between two geopoint fields.
- Added the enclosed-area() function to calculate the area enclosed by a series of three or more geopoint fields.
- Added support for self-hosted SurveyCTO Server licensing, for cases where organizations wish to host their own servers.
v1.21 also included the following small changes:
- Changed the server to reject .zipx file attachments since Android devices don't support the ZIPX format.
- Fixed a problem where downloading the same form onto a device multiple times -- with file attachments -- would result in a cryptic "ENOENT (No such file or directory)" error.
- Changed the indexed-repeat() function to suppress errors when passed instance numbers are invalid, to match the behavior of v1.17. When the instance number is invalid, an instance number of 1 will be automatically used instead (returning the first instance).
- Fixed a rare problem where canceling a slow download of a form's attachments could result in an attachment-less version of the form installed on the device.
- Fixed the new calculate_here field type to properly trigger reconsideration of field relevances when recalculating.
- Fixed the new calculate_here field type to not always have one decimal place included (.0) after integer values.
- Fixed a problem where saving a form submission multiple times would lose any comments already attached to the submission.
- Broadened compatibility of Collect's support for auto-downloading form updates, to function properly on devices with older versions of Android.
- Fixed the Russian Collect UI to appear for the Russian locale (rather than the Romanian locale).
Version 1.20 included the following improvements:
- Added the ability to calculate default values for form fields (which allows for, among other things, pre-filling fields with pre-loaded .csv data).
- Updated Collect to only show the latest version of a form in the Fill Blank Form list, and added the option for Collect to automatically download form updates in the background. Simply go into Collect's General Settings, then check Auto download with Wi-Fi and/or Auto download with network (the latter being to download via 2g or 3g cellular networks). When enabled and the appropriate connection is available, Collect will check hourly for form updates; updates will be downloaded automatically, but they won't be installed until the user confirms the installation by clicking on the Install Form Updates button (which will appear on the main menu when updates are available), or by confirming the update when about to fill out an older version of the form.
- Added integration with Enketo Smart Paper for the ability to fill out some survey forms with any modern web browser.
- Added quick-reference help tabs into downloaded form templates and all sample forms. These are simply additional worksheets in the Excel workbooks.
- Added the constraint builder, constraint tester, relevance builder, and calculation builder to the client, so that they could be used offline. To use one of these tools from the client, simply select it from the Tools menu's Form tools... sub-menu. (Also made it easier to perform offline form validation on Windows, removing the requirement that Python be installed.)
- Made a series of improvements to the SurveyCTO Collect user interface: updated the French and Spanish localizations, so that the interface is fully and correctly translated; added a second confirmation prompt for when somebody tries to exit a form and Ignore Changes; made common Internet connection errors a bit more clear; added the form ID and version to all form lists; and updated the Go-To Prompt interface to show field names for fields without labels.
- Made a series of improvements to the printable versions of survey forms: added the option to include constraint and/or relevance expressions; added support for images and field hints; added indication of required fields; added support for optional text to appear in the response area (to facilitate the filling-out of printed surveys); and improved the display in cases where multiple-choice options are dynamically filtered.
- Dramatically improved the performance of SurveyCTO Collect's mid-survey save operations by exhaustively checking and cleaning form data only when finalizing a form. If you wish to keep checking and cleaning data on non-finalizing form saves, simply un-check Enable quick saves in Collect's Admin Settings.
- Improved the performance of SurveyCTO Collect when first initializing pre-loaded .csv data.
- Added the ability to easily send crash reports from SurveyCTO Collect. If the app has crashed, click the device's menu button from the main Collect menu, select Admin Settings, then click the device's menu button again and select Submit Crash Report to send a report directly to SurveyCTO support staff.
- Added the calculate_here field type for calculations that only happen when users reach the field's place in the survey. (In contrast, regular calculate fields calculate whenever forms load or save, and whenever fields upon which they rely change.) For example, to capture the date and time at which a user first reaches a particular point in your form, use a calculate_here field at that point, with "once(format-date-time(now(), '%Y-%b-%e %H:%M:%S'))" in its calculation column.
- Added an explicit option to retain old media and .csv files when uploading a new version of a form.
- Added the once() function for calculations that should only happen once (as in, e.g., "once(random())" to draw a random number once and then keep that same draw as the form is edited, saved, and ultimately finalized). If a calculate expression is not enclosed in once(), it will recalculate periodically, including each time that the form is edited and saved.
- Added the username() function to store the currently-configured Collect username in a form.
- Changed terminology from "client name" to "server name" for the unique server name assigned to each SurveyCTO customer (as in servername.surveycto.com).
- Fixed Stata .do file templates to filter out duplicate multiple-choice values (for when cascading-selects are being used).
- Fixed Collect not to crash when, for an image field, the user selects an existing image from certain problematic apps (e.g., the multi-tabbed selector app on the Google Nexus 7).
- Fixed forms to upload without errors when they use position(..)-1 within a calculate field that may not be relevant.
- Fixed Google Earth export options in the client, to support the selection of forms in which GPS fields are within or after field groups.
- Made other small fixes and improvements to error messages.
Version 1.172 was a very minor maintenance release. It included only one fix, for an error that would appear, starting in 1.17, on form fields that had no labels (only images).
Version 1.171 was a very minor maintenance release. It included the following small changes:
- Changed the default formula for form versions (in form templates and samples) to update the version every minute. With the old formula, if you updated a form within 15 minutes of the last change, it would be difficult to upload (because the version number would not have automatically incremented yet).
- In the printable version of forms, added support for references in group labels and began showing the group name when no group label is present.
- Fixed it so that a help search with an underscore in the search string would work properly.
- Fixed Stata .do file templates to refer to the proper .csv filenames when loading repeat-group data with the "include group names in .csv column headers" option disabled.
- Changed the client to be more conservative when it encounters a new version of a form with the same version number as the old version: now it will assume major changes to the form, back up the old data, and start fresh with the new form definition.
- Fixed it so that old versions of form files are not shown when you select to download form files.
- Updated the client to use the latest "include group names in .csv column headers" setting whenever generating Stata templates -- even if the setting has just been changed and not yet committed.
- Fixed Collect to properly display field references in field hints when those hints are translated into multiple languages.
- Fixed Collect to properly display hyperlinks after hyperlinks to empty fields.
- Fixed Collect crashes when swiping forward before a form is ready and when a submission fails to send with the French locale.
- Fixed broken Browse buttons in the Mac version of the client.
Version 1.17 included the following new features:
- New export options for data analysis and visualization. In the client, all export options are now located within Preferences; new options support exporting subsets of form data to Google Earth (.kml files), to www.StatWing.com, or to separate .csv files (for use in other data analysis or visualization tools).
- New Collect option to hyperlink field references in notes, question labels, and hints. When enabled, field references will be clickable so that users can jump back to quickly change earlier responses.
- New server option to download printable versions of survey forms. Printable versions can include explanations about relevances and constraints, and can be saved and edited in Microsoft Word.
- New comments field type to allow users to add free-form comments to forms, as they fill them out.
- New client option to validate forms while offline.
- New server option to download all original form files from the server's administration console. On the Forms tab, just choose Download form files for any form, and you will be able to download the original form definition (in spreadsheet or XML format) as well as any media or .csv attachments.
- New client option to export data without group names in .csv headers (so that, e.g., no group names appear in Stata variable names).
- Ability to test new forms without them showing up for most users. Just begin any form title with "TEST - " and it will not appear, by default, to users of Collect or the client.
- Improved Excel template for new forms, including conditional formatting and a formula for the form version (which will automatically increment every 15 minutes). This new template is in .xlsx format; if you are using a pre-2007 version of Microsoft Excel, you may need to install the MS Office compatibility pack.
- New version() function for recording the current form's version number in a calculate field (just put "version()" in the calculation column).
- Richer support for external apps. (This feature is effectively invisible. It allows for more form fields to be more easily passed to and from external apps; it will allow for new integrations with custom-written apps and other apps that support "external intents.")
v1.17 also included a few small fixes and some improvements to error messages. For example, some common error messages now hyperlink directly to relevant help topics, in order to make those errors easier to correct.
Version 1.163 was primarily a minor maintenance release. It included a few small improvements:
- Added support for nested repeat groups in Stata .do file templates.
- Added support for images in .csv-based multiple-choice fields.
- Added the option to export .csv files with some column separator other than a comma.
- Improved a number of error messages to be more clear and user-friendly (e.g., on form upload, or when referring to a repeated field from outside its repeat group).
- Added progress feedback into the user interface for some long-running processes (e.g., when loading, saving, or navigating within large forms in Collect, or when waiting for Google Drive on the server).
v1.163 also included a series of small fixes:
- For very large forms being repeatedly retried over poor connections, sometimes the server would end up with duplicate records for certain submissions. This would result in "SUBMISSION NOT RETRIEVED" ("Fetch of a submission failed. Detailed error: Internal Server Error (500) while accessing...") errors when downloading these submissions. SurveyCTO support staff would manually correct these errors; now, the errors will no longer appear in the first place.
- For large numbers of very large form submissions, the client could run out of memory performing mail-merges. We have now enabled the client to use substantially more memory.
- We have fixed the jr:choice-name() function to work for multiple-choice fields that are filtered based on earlier responses.
- When pre-loading data from .csv files with blank lines, Collect will no longer give errors (it will simply ignore blank lines).
- Fixed Stata templates to ignore line-breaks and convert double-quotes in multiple-choice option labels.
- Further improved the ability to automatically and manually resume after a crash in the middle of filling out a form.
- Disabled the form-upload warning that would appear whenever using the disabled column in a form definition.
Version 1.162 was a very minor maintenance release. We discovered a small vulnerability in Collect: if you canceled in the middle of finalizing and saving a very long form, then moved away from that form and back again, you could experience a crash; further, in this case, Collect did not auto-resume the form so that it could be fully saved.
Our fix was in two parts:
- We fixed the vulnerability in the cancel-save process.
- We also added a new "Resume Crashed Form" option to the main Collect menu, accessable via the device's Menu button. Via this option, users can resume old forms even in cases where Collect did not auto-resume. (We frankly expect nobody to ever need to use this option, but we added it out of an abundance of caution.)
Version 1.161 was an emergency maintenance release, to solve a problem with Collect's form encryption on new Android 4.3 devices. All versions of Collect prior to this v1.161 release slightly corrupted encrypted submissions, causing errors on decryption.
Our fix was in two parts:
- We fixed Collect to no longer corrupt submissions.
- We also fixed the client so that any submissions corrupted under earlier versions of Collect would be automatically fixed, decrypted, and exported. This auto-fix technology was added to avoid the possibility of data loss for any client who might have already collected encrypted data with an older version of Collect (on Android 4.3).
Version 1.16 included the following new features:
- Integration with Google Drive for form uploads and downloads. This means that forms can be collaboratively designed directly in Google Spreadsheets, without the need to export to .xls or .xlsx format. Find the new Google Drive features in the Download form template and Upload form pop-ups, on the Forms tab.
- An improved help system, including 94 screenshots, walk-throughs of every sample form, and the ability to search help content.
- Added new functions for repeated fields: sum(), min(), and max().
- Slightly improved support for nested repeat groups. (There had been a Collect bug that affected certain types of nested repeat groups when they used fixed counts.)
Version 1.15 included the following new features:
- Graphs that plot form submissions over time. See these on the new Statistics tab.
- Improved and documented support for collecting annotated pictures, drawings, and respondent signatures. Use by simply including "annotate", "draw", or "signature" in the appearance column of an image field.
- Automatic hyperlinking of media and audit files in mail-merge output.
- New Collect setting to defer form validation until the end of a form. When the new constraint-processing option is set to "Defer validation until finalized", field constraints will only be checked when users go to finalize their forms. (For most projects, the original, default behavior is what is desired, however: you do not want users to be able to swipe forward until a field's constraint has been satisfied.)
- Improved user interface for the Forms tab.
v1.15 also included an important fix for a bug introduced in v1.14: if saved forms are edited such that they shrink in size, errors will no longer result. Other, smaller fixes have been included, mostly from the ODK community (for example, a problem downloading images within repeat groups has been fixed).
Version 1.14 included the following new features:
- Ability to configure the client to automatically download and export data periodically, in the background. From the client, simply choose Schedule sync... from the Tools menu.
- New random-once() function to generate a random number that will not change if a form is later edited.
- Ability to re-encode .csv files to UTF-8 format, so that different fonts will display properly from pre-loaded .csv data. In the client, just select Re-encode .csv file... from the Tools menu.
- Added the jr:choice-name() function, to be able to reference the label for any prior select_one or select_multiple choice. On the Forms tab, choose Build calculate expression..., then choose either Label for select_one selection or Label for select_multiple selection.
- Improved memory usage for large forms. Collect will now require 80-95% less memory to load any given form.
- Improved save-point performance and reliability. Collect saves the current form every time you swipe from one question to the next, so that it can automatically resume if there is a crash. Now, these save-points work more reliably in the case of hard Android or device failures, and the performance has been dramatically improved by moving the save-point process into the background (so that it saves the save-point while the user is looking at the next question, rather than making the user wait mid-swipe).
- An improved Collect user interface for newer Android devices, in order to conform to newer Android UI standards. Context-specific menu options now appear as buttons in the top-right, as part of a title/menu bar along the top of the screen.
- Support for references in media columns, so that image, audio, and video filenames can be specified dynamically.
- Improvements to error reporting when form definitions are rejected on upload, so that it is easier to find and fix errors.
v1.14 also included a series of small fixes: Collect will no longer crash when the device's SD card is mounted or un-mounted; use of selected-at() will no longer trigger validation errors; Collect's overall list of form fields now properly lists fields following a repeat group; the client can now download encrypted forms that are incomplete, provided that they have been marked as complete; and newer Android devices will no longer have problems with attached files sometimes failing to download with form definitions.
Version 1.13 included the following new features:
- Ability to view and manage incomplete form submissions. On the Forms tab, incomplete submissions are now noted, along with an option to view and accept them.
- Option for Collect to download compressed form XML. If you have very large forms that take a long time to download, you can now check the Compress form downloads option in Collect. When enabled, the server will zip all form XML files, and Collect will download the zipped versions and unzip them automatically.
- Option for the Client to also download compressed form XML. To enable, just check Compress form downloads in the SurveyCTO Client's preferences; like in Collect, zipping and unzipping is fully automatic.
v1.13 also included a fix for OSX-zipped support files, small fixes to Stata templates (for mixed-case field names and different permutations of group nesting), and small improvements to help content.
Version 1.12 included the following new features:
- The ability to search-and-select pre-loaded .csv data. As of v1.12, any select_one or select_multiple field can pull some or all of its choices from a pre-loaded .csv file. These dynamic choice lists can depend on previous fields, or even display the results of search queries.
- Ability to compress form support files in .zip archives. Just compress large support files -- such as pre-loaded .csv files -- and upload them as one or more .zip files; in Collect, they will be automatically unzipped.
- Support for longer, more complex forms in Collect. Now, on newer devices (Android version 3.0 and up), Collect will be able to utilize more of the device's memory, and so will support larger forms without crashing with out-of-memory errors.
- The ability for later repeat groups to refer to values from prior repeat groups.
- Improved Stata templates, so that extraneous variables will be dropped for repeat groups inside other groups, and so that repeat groups with over 10 instances will sort in the correct order.
- Small improvements to error messages when there are errors in the choices sheets of uploaded forms.
- Addition of form-upload warnings for potential form issues that do not prevent the form from being uploaded and used.
- Improvements to the documentation of select_one and select_multiple appearance styles.
- Correction for the substr() function, so that it is easier to use in expressions (without triggering validation errors).
- Addition of the format-date-time() function, so that both dates and times can be captured and expressed.
Version 1.11 included the following new features:
- Ability to pre-load data into your forms, from baseline surveys, listing surveys, and other sources.
- Ability to generate calculate field expressions with a simple Calculation Builder (find as an action on the Forms tab).
- Ability to reference current index number within repeated groups (search for "position" on the Forms help page).
- Improvements in UNICODE support in form titles and user responses.
- Improved client installation process for Windows.
- Improved client installation process for Mac systems with gatekeeper restrictions.
- Dramatically faster load time for server Forms tab when there are many forms and thousands of submissions.
- Small improvements to Stata templates.
v1.11 also included the latest ODK v1.3 features, including:
- Ability to auto-send finalized forms when network connections are available (see General Settings in the new Collect).
- Ability to restrict the features accessible by users of Collect (with password-protected configuration).
- Ability to easily export and copy Collect configuration, so that it is easier to set up lots of devices.
- Ability to auto-name form instances based on survey details.
- Ability to configure Back and Next buttons in Collect, as an alternative to swiping forward and backward (see General Settings in the new Collect).
- Ability to see count of finalized and saved forms on main Collect menu.
- Improved compatibility for newer Android devices.
Version 1.10 was the first public release of SurveyCTO. It included a wide range of improvements over earlier releases, each of which had been deployed in a limited number of pilot sites.