Tackling TRIM and SharePoint at AFMA

By Thomas Kaufhold (AFMA) and Paul Taylor and Nigel Carruthers-Taylor, iCognition

Implementing an Electronic Document and Records Management System (EDRMS) at the Australian Fisheries Management Authority (AFMA) appeared on the surface to be a typical and unremarkable task. However the project actually differed in the unique opportunities it presented, the engagement of the users and the commitment of information management staff to boldly work in uncharted territory.

AFMA’s move to an EDRMS, undertaken with the assistance of information management solutions provider iCognition,  was a combination of the need to meet the Australian Government’s Digital Transition Policy and the desire to improve information management within the Agency. These improvements included Business improvement through improved discoverability, sharing of information and improved document version control; the ability for all records to be stored, managed, and retrieved compliantly; and for AFMA staff to understand and meet their recordkeeping obligations easily and transparently.

AFMA used HP TRIM to manage their hardcopy records as well as network drives and Outlook for their unstructured digital information. These legacy digital tools had the typical deficiencies, including inability to access information based on organisational structure, lack of version control and incomplete discovery.
A scoping study of information and records practices was undertaken which identified issues and requirements taking account of the Digital Transition Policy.

Governance first
The project approach was to ensure that the governance foundation was updated and improved prior to any decisions being made with how the technology would be implemented. This meant that a number of decisions had to be made in both the records management area and with the ICT environment.

The user experience was also a key consideration to the decision.  The goal was to ensure that the solution would be easy to use and require minimal training to use.

The biggest historical barrier to the uptake of EDRMS and records management compliance has been user confusion about what they need to do, why and when. A key outcome of the project was to change behaviours in preparing sharing, distributing and storing information.

When the project commenced, SharePoint 2010 was being trialled to replace group drives.  It was being utilised by AFMA as their default development application. HP TRIM 7 was also used in the AFMA environment to manage physical records.

Consideration was given to the possibility and benefits of upgrading both SharePoint and HP TRIM to later versions,  and later, integrating them as the document development and storage interface.
AFMA’s Standard Operating Environment (SOE) did not preclude an upgrade, but the implementation decision was to base the EDRMS on SharePoint 2010 and HP TRIM 7.3.

Key to this decision was SharePoint’s difficulty in complying with Australian Government legislative, accountability, and mandated records management requirements. Government assessments have concluded that SharePoint is not a recordkeeping system and must be configured to provide or extend certain recordkeeping features and functionality.

Even with the release of SharePoint 2013, the gap between native SharePoint functionality and the legislative requirements to manage records is substantial. AFMA chose not to modify SharePoint to meet the recordkeeping requirements, as Government assessments note that care should be taken to avoid customising a SharePoint implementation to ensure ongoing product support and maintenance.

SharePoint + TRIM
AFMA therefore chose to cover the records management gaps using HP TRIM SharePoint Integration. Not only was it planned that the HP product would ensure the SharePoint environment was compliant to Australian records management standards, but it was hoped that the total cost of ownership of SharePoint (which has been estimated at $49 per user per month) would be reduced by controlling and managing the information within SharePoint using lifecycle management policies.

The aim was to establish a SharePoint environment that all staff were comfortable using, then integrate the HP TRIM EDRMS into the backend for seamless records management to allow the user to access the full capabilities of SharePoint while records are captured seamlessly and information is well governed across the enterprise.

The integration of SharePoint with HP TRIM was christened 'SQUID'. The name is a mnemonic based on the principles of:

  • Secure – safe and secure information
  • Quality – quality information you can trust
  • Usable – usable information you can access
  • Information – information is managed and governed appropriately
  • Digital – achieving better business outcomes through digital management in a digital world

Records or Information?
SharePoint to HP TRIM integration can save documents to TRIM in a variety of ways. Most commonly it is performed manually, by a timer job, by action or automatically. To simplify the user experience, AFMA has removed the differentiation between a record and information. All unstructured information would be saved into the ERMS solution automatically.
This approach presented less risk to AFMA as all unstructured information would be managed as records and users would not need to determine if and when to create a record. This was a key advantage for both management and users.

The usual changes and updates were made in AFMA’s information governance. The Records Policy was supplemented to include handling of electronic material. A new Records and Information Management Framework was issued in July 2014 and the information management principles contained in the framework were key to creating the necessary policies and procedures for the EDRM solution.
SharePoint enables access to information quite differently to a typical EDRMS such as HP TRIM. This difference was a primary consideration in creating and configuring the AFMA solution.

SharePoint is more akin to an Electronic Content Management tool where information is typically managed by Sites and Sub-sites with documents placed within Document Libraries. Other SharePoint content such as Lists, Wikis and Blogs are also managed in a similar way to the Document Libraries (DL). Of course there are many ways to use SharePoint and documents do not have to be placed in a Document Library.
Most SharePoint implementations are site driven. Sites usually have owners and these owners can apply a variety of levels of access to users or groups. The governance of SharePoint is generally left to the site owners to handle and is rarely managed at solution level. The project was in a fortunate position as there was initially only a small number of SharePoint users. SharePoint was therefore able to be designed in the way believed was most appropriate for AFMA.

Because of the issues AFMA had with its existing systems and the established principles, it was critical that governance was managed from the top down. Through this means, and in conjunction with improving discoverability, sharing, etc., a hybrid information structure was created that was based largely on function, as opposed to organisational hierarchy.
The structure is a hybrid one because it is not purely functional nor is it just an organisational view.

In practice, three Site Collections were created. Two were for  Branches of AFMA and the other was an all-AFMA site. The Branch Site Collections purpose was to capture any functions information that was unique to that Branch. The all-AFMA Site is used to capture all other information.

Each Site and Sub-site related to a function. Document libraries equated to a container or electronic file in HP TRIM. For example, part of the all-AFMA site collection would be all-AFMA – Finance – Travel – Acquittals FY 2014-15 – Individual documents.

This approach differs to traditional SharePoint implementations where similar information can be stored in various logical locations.
Metadata fields are added to documents to provide further detail to the information, including user visibility of the information owners.  Metadata in a SharePoint Document Library is particularly useful as it can be used to visually filter results in a similar way to an Excel spreadsheet.

Achieving Compliance
The decision to have a Document Library directly equate to a HP TRIM folder had compliance benefits and implications such as:

  • Lifespan – document libraries should have a limited lifespan so they can be disposed of in accordance with the Archives Act 1983;
  • Name – the name of the DL should reflect the contents; and
  • Contents – periodically monitor the contents of DLs to ensure that their disposal class remains appropriate. The link feature can be used to better ensure that information is more discoverable and logically available in multiple locations.

Other configuration changes were made to improve compliance without impacting on the user experience. The most prominent of these are delete and copy functions.

The  users are not able to delete information from SharePoint because it would be confusing to have information in HP TRIM and not SharePoint. It also better maintains the integrity of the information in SharePoint. The solution was to create a workflow whereby users could remove the document from everyone’s view by clicking a button and writing a small justification for “hiding” the document. The hidden document is then flagged for the Information Management team to review and action from a maintenance perspective. SharePoint’s concepts of move and copy differ from those of a typical EDRMS. SharePoint allows users to create copies of documents in multiple locations. They can also choose to associate these documents, so that when one is amended the other will be updated. More importantly, they can choose not to do this.

Copying goes against the principle of a single source of truth.  The function was therefore disabled and users are only allowed to create links rather than copies. The users actually preferred this, and it did not restrict any business processes.

User Interface principles
A set of user interface principles were developed aligned with the overarching principles but with a focus on the user experience. They are:

  • Quick access to information – SharePoint as the user interface for content management should result in ‘reduced clicks’ (keystrokes) for users. The intent is to encourage user uptake of the system by reduced complexity of interaction with the application;
  • Single source of truth – Information should only be stored in SQUID once. The interface allows for the use of links, meaning that one version of the information can logically appear in more than one location, without actual duplication;
  • Functional management of information – the information structure of SQUID is based on a functional method of storage, rather than an organisational approach. This will help ensure that similar information is stored together and that it will be more flexible in responding to organisational change.
  • Streamlining and improving user interaction - there should be no unnecessary prompts through the process of creating and saving content to a document library. Although this is configuration-related, it also includes trying to reduce the instances of possible errors that users may face in using the application.
  • User interface provides easy search for content - Searching within SharePoint should allow for the retrieval of content that is stored exclusively within HP TRIM, and allow users to expose this content into SharePoint if required.
  • Simple navigation – This means that SharePoint sites in SQUID should have a specific business purpose and not exist to try and replicate network folder navigation or structure.

For the everyday user the main changes were the move to the SharePoint interface and the functionally-based arrangement of information. They gained significant improvements in information access and sharing, the ability to create on-demand reports through the use of metadata, and more advanced features such as concurrent document editing and version control.

All of these features were either unavailable or ineffective when using network drives. A “Super User” model was implemented to maintain good information practices and engender a sense of ownership of information structure. The Super users are responsible for requests to create or alter existing Document Libraries and sites. They also create SharePoint views, alerts and add announcements for these locations. 
Finally, Super users have responsibility for ensuring the consistent use of metadata within their area(s) of SharePoint. Metadata and other metadata needs are also addressed in a whole of Agency manner through regular Super User meetings.

SQUID went live in mid-October 2014 to about one third of the AFMA staff. In preparation for the implementation Super Users were established in the work areas to assist staff in using SQUID and to liaise with the Share Point Administrator and Records Management staff (RMU).  The main role of the Super users is to vet requests for new document libraries (DLs) and to suggest/request new Share Point sub sites, changes to sites and other enhancements, such as establishing metadata sets.

During go-live training was provided to all users and the RMU and Standard Operating Procedures (SOPs) were provided for users and Records Management staff to use when creating Document Libraries (DL) and naming documents. 

As a result of the implementation of SQUID, email protocols have been developed to ensure emails are saved by the appropriate person and help reduce duplication. The handling of scanned information was also addressed through new procedures.

Documents that have been scanned into SQUID are stored by the RMU and are destroyed in accordance with the Normal Administrative Practice Policy under the Archives Act 1983. A SharePoint email integration tool is currently being considered.

The implementation of SQUID meant that Records Management staff had to learn SharePoint and new procedures. They became more proactive in dealing with users and engage in more problem solving activities at an information management level.  They also contribute to a constant process of fine tuning and updating these procedures. 

RMU staff responsibilities have been expanded and one of the main additions is the quality assurance of the creation of Document Libraries to ensure they are adequately titled, determine and implement security protocols for accessibility purposes and map the DL to the Records Authorities for disposal purposes.  Once the DL has been quality assured it is released for use. 

RMU staff have taken over from the project staff communications and now attend monthly FMB meetings to update users on changes to SQUID and to answer questions or concerns. There is also a regular Super users also met to discuss operational issues and suggest improvements to sites and processes.    

Constant communications occur between Records Management staff and SharePoint Administrators to discuss and decide on requests for DLs, enhancements and operational issues raised by users.  As issues are addressed the regularity of discussions has reduced.  When the solution went live there was significant activity in the first month to create DLs and uploading of documents.  This gradually decreased as the initial need for DLs was satisfied and required documents were uploaded. The creation of DLs has now settled down to accommodate new business.

SQUID has become a business as usual now for FMB and the RMU and SharePoint team work with them to provide additional functionality and improvements to meet their business needs.

What did we learn?
There were three major lessons that we took away from the project.

  1. Don’t assume SharePoint is well understood by users. The project spent more time than expected explaining the interface and capabilities of SharePoint. The need for users to understand how SharePoint operates was critical throughout the entirety of the project.
  2. Ownership of knowledge transfer. The integration of SharePoint and HP TRIM typically crosses over two or more teams within an organisation. Questions of ownership and responsibilities need to be addressed as soon as possible in a project. Ideally prior to any work commencing. To achieve maximum benefit, there is a transfer of knowledge between teams. SharePoint developers will need to understand elements or records management and information management specialist need to embrace technical SharePoint knowledge.
  3. Align your technical environments. This is a common refrain in technical projects but it cannot be underestimated. Your Development, Test, Training and Development environments need to be as consistent as possible. Any discrepancies can cause surprises, delays, rework and can make bug hunting much more difficult.

The journey at AFMA is not complete and there are a number of priorities on the immediate horizon such as adding new user requirements to SharePoint, adding a third party tool to improve email management and rolling out to the remainder of the Agency.

However, AFMA is confident that it can meet current and future challenges as the information management foundation is sound.
With good governance, compliance and principles in place, the technical solution is extensible and user focused and, most critically, providing useful and practical solutions to business problems.

Thomas Kaufhold is Records and Office Manager at Australian Fisheries Management Authority, Nigel Carruthers-Taylor is the Principal & Director at iCognition, who provided strategic advice for the SQUID project. Paul Taylor was the iCognition project manager for the project delivery.  Contact nigelct@icognition.com.au  for further information.