Ver. 2020.5.4 - further SharePoint extensions

  • Further improvements when processing SharePoint locations with more than 5000 items
  • Fixed bug that PowerQueries sections was ignored when Excel documents were in SharePoint.

This upgrade is recommended if you have document locations with more documents than SharePoint threshold value (default: 5000)

Ver. 2020.5.3 - additional SharePoint support

New in version 2020.5.3 (13th of May, 2020) - ...

  • Extended "Add SharePoint site" with option to set LegacyAuthenticationMode directly.
  • Extended "Add SharePoint site" with option to jump to SharePoint configuration directly.
  • Changed coding to identify maximum document ID (relevant for SharePoint) to avoid threashold limits from SharePoint
    This upgrade is recommended if you have document locations with more documents than SharePoint threashold value (default: 5000)

Ver. 2020.5.2 - a lot of new features in ReplaceMagic

We hope that you are all safe and healthy managing to balance your private and business life in those difficult COVID-19 times.

In last 2 months we spent most of the time helping customers overcome current situation but ReplaceMagic team also actively worked to prepare new version will of new and improved features.

Version 2020.5.2 has only correction not to show error message when ReplaceMagic configuration is saved (compared to 2020.5.1).

Below you can find what is new:

    • Trial of 30 days is only for replacement process. Scanning of documents and check for broken links will not have any time limitation (or any other limitation).
    • From next release ReplaceMagic version numbering will change to format YEAR.MONTH.RELEASE -> something like 2020.5.1 which is first release in May. In case that there is new version in May next number will be 2020.5.2 or if next version is in June number will be 2020.6.1
    • Scan documents result table extended with DocIdUrl link (SharePoint only).
    • PowerQueries in Excel - ReplaceMagic supports changes of source in Excel PowerQueries:

 

  • During generation of XML configuration for command prompt run all XML reserved characters are encoded to enable proper XML formatting. During start of ReplaceMagic with XML configuration file encoded characters will be decoded back to what you initially provided.
  • Extended logging of errors during test of SharePoint connections. Options "Create log file" and "SharePoint developer log info (enable only on ReplaceMagic team request)" from Configuration => Log have to be enabled.
  • During preparation for scanning of SharePoint location it is required to find maxID of documents in selected location. To accelerate identification process we included parameter MaxID range factor:

    Default value is 2 meaning that upper limit everything when there are no documents in given range will increase according to the formula => NewUpperLimit = UpperLimit + 2*UpperLimit. As this sometimes might not be enough or is too much you can adjust this parameter. By setting lower than default of 2 preparation phase will take a bit longer or if you set higher factor number you might reduce the time. From our experience, value of 2 make usually most of the sense and should not be changed.
  • Support for email document types MSG and EML - Added support for email formats EML and MSG - ReplaceMagic supports changes in hyperlinks in email message body


    Email functionality is part of ReplaceMagic.Ultimate edition.
  • To be able to process documents ReplaceMagic needs access to ...\users\CurrentUser\AppData\Roaming\ReplaceMagic or ...\users\CurrentUser\AppData\Local\ReplaceMagic or ...\ProgramData\ReplaceMagic folder. In case that needed content is not there ReplaceMagic will automatically copy required files from installation folder (usually C:\Program Files (x86)\ReplaceMagic). If this is not possible you will get notification similar to: and scanning or replacement process will exit. It is important to grant permissions to user running ReplaceMagic to be able to create folders and write into them.


  • With parameter "List all skipped documents in Additional info tab" ReplaceMagic will show all documents that are skipped during processing including if document extension was not on list of selected extensions or if document create, last access or last modified dates are not in selected range, etc...
    Previously, ReplaceMagic was showing only skipped documents during real document processing (scenarios like: document is corrupted, document is locked by somebody, document was password protected, etc...).
    To enable this setting go to Configuration => Processing as select "List all skipped documents in Additional info tab":
  • Support for SharePoint not-LegacyAuthProtocols - Added option to select between SharePoint modern and legacy authentication protocols (SharePoint parameter: LegacyAuthProtocolsEnabled):


    In case of this setting on True (default value) you will need to ensure that  SharePoint parameter LegacyAuthProtocolsEnabled is set to true (see ReplaceMagic help article SharePoint MFA (multi-factor authentication). As default setting on False as this setting will work in both cases. If you want you can enable legacy mode by setting it to true. Remark: In case that you have files with % in name when this parameter is set to False ReplaceMagic will not be able to save document after replacements are done and those documents will be shown in Additional Info tab as skipped documents. When legacy mode is set to True ReplaceMagic will not have this constraint but before setting it to True please check that your admins are allowing it. Depending on security policy this parameter might be disabled in your SharePoint environment.
  • Multi-factor authentication added - We added in Beta mode Multi-factor authentication mode to form where you can select where you want to connect on SharePoint side:

    More details about this authentication mode can be found in ReplaceMagic help file under SharePoint MFA (multi-factor authentication)
  • We added additional logging features to measure how time is split during scanning and replacement in documents stored on SharePoint. To enable this option go to Configuration => Logs  and select parameter "SharePoint performance markers"

    After this parameter is enabled in your logs you will see entries like (without comments in bold; btw. this values were when SharePoint was throttling our requests; without throttling we see 4-10 faster performances):
    • Binary Load Document: document.docx: 2143,2041 ms (comment: this is where we download documents from SharePoint)
    • Preparation for Meta Data processing: document.docx: 1270,1174 ms (comment: access to the SharePoint)
    • Replacement process: document.docx: 0,9958 ms(comment: processing done on your computer)
    • CheckIn/Out or Publish/Unpublish update: document.docx: 2007,0992 ms (comment: access to the SharePoint)
    • Preparation for save: document.docx: 89,7238 ms(comment: processing on your computer)
    • Upload to SharePoint: document.docx: 3177,0676 ms (comment: access to the SharePoint)
    • Set document Metadata back: document.docx: 16490,9978 ms (comment: access to the SharePoint)
  • Performance improvements of findMaxID functionality for documents stored in SharePoint
  • Time split when processing documents on SharePoint - Extended result dashboards to include information about processing time split when working with SharePoint. To activate this feature you will need to go to Configuration => Logs and select option "SharePoint performance markers":

    More infos in ReplaceMagic help under: Scanning dashboard & Replacements dashboard
  • Improved error handling if ReplaceMagic databases are not available.
  • Added option to jump to SharePoint configuration directly from form where you can create SharePoint configuration:
  • Included reminder to select only really necessary options for SharePoint processing to avoid unnecessary SharePoint workload.
  • Optimized processing when no SharePoint options are needed to reduce unnecessary calls of SharePoint.
  • ReplaceMagic Themes - from this release you'll have possibility to change look of ReplaceMagic. Under Configuration you can find option "ReplaceMagic Theme"

    Where you can select different design themes for ReplaceMagic installation.
  • SharePoint App-Only authentication mode - SharePoint App-Only authentication mode - we added App-Only Authentication [BETA]

    To use it you will need to create Client ID and Client Secret. Description how to do that can be found in article "Granting access using SharePoint App-Only"
  • Remarks about SharePoint Throttling and current COVID-19 situation => more details in ReplaceMagic help file
  • ReplaceMagic help file reorganized to group most of SharePoint topics under one area
  • Fixed bug that error message was shown when saving ReplaceMagic configuration if ReplaceMagic Theme was not set
  • Changed title "SharePoint Markers" to "SharePoint Metrics"
  • Added option in About form to see validity of trial version
  • Changed default name of Scan Documents export from "Find Documents..." to "Scan Documents"

Ver. 4.8.1- usage of SharePoint docID instead of uniqueID

What is new in version 4.8.1:

  • Minor change: Instead to use uniqueID of SharePoint documents we are showing in result lists now documentID. Document ID Service has to be enabled to use this feature.

Ver. 4.8.0 - change SharePoint list item hyperlinks and more

What is new in ReplaceMagic 4.8.0:

  • Extended SharePoint functionality that multi-select SharePoint location works. Use Ctrl+Click or Shift-Click to select multiple locations at once.
  • Improved error handling during registration process.
  • Extended About form to include current ReplaceMagic version information.
  • Extended error handling during selection SharePoint locations for scanning/replacement.
  • Added UniqueID in case of documents stored on SharePoint. This ID can be sent to SharePoint to access file without knowing document location.
  • Fixed bug when major/minor versions were used and we try to unpublish already unpublished document.
  • Fixed issue with progress bar when SharePoint structure was expanding.
  • Added sorting of treeview showing SharePoint structure.
  • Provided info how to use ReplaceMagic on MFA enabled SharePoint site. Check help file for more details.
  • Fixed that SharePoint documents are opened after clicking in ReplaceMagic grids (previously, not proper URL was set so documents could not be opened).
  • Improved performances when expanding SharePoint tree view structure.
  • Added column "Creation Date" in scan documents result table.
  • Filtering on creation, last modification, last access date changed to work with BETWEEN operator by setting start and end date.
  • Exports extended to include all added field. In case that you had export which you are using for later import you might need to add additional field (they can be with empty value).
  • Added support to scan SharePoint List Items for hyperlinks. Remark: ReplaceMagic will not check if native SharePoint URL fields (btw. they are controlled only by SharePoint) in document library are broken or not. Those fields are having links like: DocIdRedir.aspx?ID=JS24VZ4MQFT5-2144151531-14.
  • Extended ReplaceMagic to support changes of hyperlinks in SharePoint list items.
  • Check help file remarks regarding SharePoint lists configuration and impact of what ReplaceMagic can keep.
  • Fixed that click in tabs is opening processed document.
  • Improved console and logs output.
  • Fixed bug when RTD volatileDependencies.xml was lost after saving changes in Excel documents.
  • Added option to set UserAgent for checking broken links as some Web servers might reject requests without it. If this field is left empty UserAgent setting will not be used. This setting works only for WinHTTPRequest method.
  • Fixed bug that last modified date and editor were not kept after changes in Wiki/Canvas pages (content stored in SharePoint database).
  • Extended that ReplaceMagic can HTML encode/decode SharePoint pages (.aspx format) as legacy pages (like Enterprise Wiki) are encoded thus without decode is not possible to find links. As this is beta please test carefully as pages might be damaged if you make wrong replacements. To enable this setting go to Configuration=> SharePoint and select option HTML Decode/Encode SharePoint pages [BETA].
  • From this version ReplaceMagic requires .NET version is 4.6.1 or higher.

As you can see with this version of ReplaceMagic we support changes in documents stored on SharePoint but also SharePoint Wiki and Canvas pages plus also hyperlink fields of SharePoint lists. This is rounding ReplaceMagic capabilities to process links in most if not all areas of SharePoint.

 

 

Ver. 4.7.5 - major improvements command line processing (benefits for automatization)

New in ver. 4.7.5 (07th of January, 2020) - focus with this update is on Command Prompt start of ReplaceMagic with different parameters

  • License key is changed (from 4.7.3) => All customer with valid maintenance will need new license key. Send us email to support@replacemagic.com and we will generate new license file.
  • Added option to change VBA module name.
  • Added option to disable Excel formulas refresh when opening ReplaceMagic changed document. Check Configuration => Processing => Parameter "Do not refresh formulas in Excel documents".
  • During preparation for command prompt run (silent run - start of ReplaceMagic from command line with XML file) you can select that username and password to connect to SharePoint are encrypted to protect user data if somebody has access to XML file.
  • Silent run supports option to export of scan and replacements and/or to send them to email results (data in result grids in ReplaceMagic) after document processing. Default location is C:\Users\Your_User\AppData\Roaming\ReplaceMagic
  • Silent run now supports multiple configuration files. Simply create needed number of XML configuration files and start ReplaceMagic with "ReplaceMagic ConfigFile1.xml ConfigFile2.xml ... ConfigFileX.xml" and application will process configuration files one by one
  • Email Notifications - ReplaceMagic supports option to send notification email with scan and replacement results after command prompt run. To configure email system go to Configuration => Notifications
  • Default export location changed to C:\Users\Your_User\AppData\Roaming\ReplaceMagic\Exports
  • All export options (except export from Console tab) are extended to support send of export files to configured email (Configuration -> Notification)

 

Happy New Year from ReplaceMagic team

Ver. 4.7.4 - improvements in processing on SharePoint, VBA in Excel docs and more

New in version 4.7.4 (07th of December):

  • Changed parameter name "Expected max. DocLib size (default value: 150.000)" to "Max docID value". New default value is set to 1.000.000.
  • Fixed problem that when parameter "Max docID value" was set to some non-default value after restarting ReplaceMagic default value was always used.
  • Extended SharePoint developer log entries to provide more details.
  • Extended that hyperlinks in OneNote documents are reported according to where they are=> TextNode hyperlinks and ImageNode hyperlinks.
  • Fix calculation for parallelism. Previously number was defined like physical CPU x cores x logical CPUs which was wrong as number of logical CPUs were core x threads. Now ReplaceMagic will show # physical CPUs, # cores and # threads with total number like # physical CPUs * # cores * # threads.
  • Fixed periodical issues when processing VBA code in Excel files.
  • Fixed counter during SharePoint documents processing when there were more than 5000 documents in selected location.
  • Improved error handling.
  • Changed export and import functionality as a preparation for coming changes (stay tuned...).
  • Minor bug fixes and improvements.

 

This update is recommended to all but expecially users processing documents in large document libraries on SharePoint (more than 5000 documents) and processing VBA code in Excel documents.

From ReplaceMagic 4.7.3 - License key is changed => All customer with valid maintenance will need new license key. Send us email to support[AT]replacemagic.com and we will generate new license file.

 

Ver. 4.7.3 - Support for changes in VBA in Word docs + more

New in version 4.7.3 (16th of November, 2019)

  • License key is changed => All customer with valid maintenance will need new license key. Send us email to support[AT]replacemagic.com and we will generate new license file.
  • Fixed problem with usage of Regex when strings contained regular expression escape characters.
  • Cosmetic change in About form to show maintenance validity when date format was not properly converted.
  • Added tab with button to go to our document conversion cloud tool PDFtodo (www.pdftodo.com).
  • Improved error handling when processing documents.
  • Added options "Developer log info" and "SharePoint developer log info" in Configuration => Logs to enable collection of developer logs (only on ReplaceMagic team request).
  • Improved check of broken links to return real error code returned and not to show, in most cases, "404 link not found".
  • Fix bug that processing duration shown in dashboard was changing with every dashboard opening in case of time in minutes or hours.
  • Extended Configuration to show # physical CPUs, # cores and # logical CPUs.
  • Removed setting to check only limited number of links in trial version. Now, ReplaceMagic will, if instructed, always check links without any limitations (number of links to be checked).
  • Fix bug when processing .aspx pages stored in SharePoint (save was reporting Stream closed error).
  • Fixed minor bugs.
  • ReplaceMagic supports now changes in VBA code in Word documents (documents with docm extension).
  • Fixed issue that created, modified and accessed timestamp was converted to UTC instead to keep local time.
  • Fixed that when "Extended network drives" option was used no scan location was selected until not clicked in a treeview.

Ver. 4.7.2 - improvements of processing older Excel formats and VBA in Visio

What is new:

  • Further improvements of changes in older Excel formats. We recommend this upgrade in case that you have older Excel file formats.
  • Improved VBA processing in Visio documents.

In case that you have older Excel formats we strongly recommend this upgrade.

Ver. 4.7.1 - Further SharePoint, export extensions and more

New in version 4.7.1:
  1. Additional extensions added in default set of extensions => aspx, js, css. Of course, it is still possible to add custom extension
  2. Moved selection of changes in SharePoint pages (Wiki/Canvas) to separate tab in Search&Replace form. All later SharePoint specific developments will be in this tab
  3. Extended support (search&replace) for Link Sources in PowerPoint documents (for example, if you have Excel chart imported in PowerPoint presentation)

    Remark: In case that Automatic Update in PowerPoint document is not set and new link for external chart does not exist or is not accessible ReplaceMagic will make a change but chart in PowerPoint document will still be the old one. To manually update chart (pull data from new location) will need to right-click on chart and in context menu select option "Edit Data" which will open source Excel file. After Excel is opened chart in PowerPoint document will be automatically updated

    Second option is to click in PowerPoint on File => "Edit Links to Files" and there to select "Automatic Update"

  4. Adjusted export to Excel functionality to split results in multiple worksheets if number of rows is bigger than 1.000.000 (to avoid Excel limitations). There are some cosmetic issues (header row color not always the same, to be fixed later)
  5. Fix bug when excluding "Table of Content" links in Word documents
  6. Extended hyperlinks display in Word documents to include also parameters
  7. Included that fixing of broken hyperlinks in Word document covers also hyperlink parameters
  8. Added in installation information that .NET 4.5.2 is required
  9. Changed installer type and publish setup as EXE and not MSI
  10. (Expert option) - Added converter of not supported VSD,VSS and VST Visio file formats to newer VSDX, VSSX and VSTX. Until this version ReplaceMagic was not able to read and save older Visio file formats (VSD/VSS/VST). From version 4.7.1 you will have option to instruct ReplaceMagic to convert older in newer formats. To set this feature go to Configuration => Processing and select option "Convert Visio (VSD/VSS/VST) docs in VSDX/VSSX/VSTX format?"

    Disclaimer: We will always keep original Visio file format (VSD/VSS/VST) and create new file in newer Visio format. In case of network drives if you specify that ReplaceMagic should create backup beside original and new file you will also have backup file.
    In case of SharePoint only old and new file will exist
  11. Added support for Visio VSX and VTX file format
  12. Added "Proxy Configuration" to "Add SharePoint Site" form where you can provide proxy details (proxy URL, port, username and password). Only HTTP proxies are working (HTTPS does not work for now)
  13. Added option to delete list of saved Search&Replace strings in Search&Replace form
  14. Changed that double click on form used to select SharePoint location is always opening only one level below. Before complete clicked node structure was opened at once which was challenging in case that there were plenty of items in opened locations
  15. Added option to delete saved SharePoint connections from Connection History drop-down
  16. During SharePoint changes ReplaceMagic was setting after check-in/check-out, publishing, approval comment text. Now comments by ReplaceMagic can be removed or you can set your own text
  17. All documents stored on SharePoint have unique ID. Normally document IDs are created in sequential way but if some documents are deleted you might have gaps between document IDs. Due to checking numbers in steps of 5000 (previously used in ReplaceMagic) might happen that some documents were not found. To address this case we provided new functionality to find highest document ID in DocLib and to loop until this number is not reached eliminating possibility that some files are missed. As this requires that we loop until maximum number of items in document library (30.000.000 documents) to avoid unnecessary loops you can set expected maximum document library size (default value: 150.000 documents in DocLib)

    Just as a reference - scan in our test site with this parameter set to 500.000 took around 90-100 seconds (with Ultra fast scan option) and with parameter set to 30.000.000 it took around 180 seconds. In total there were only 200 documents in this site

  18. Added SharePoint DefaultCredentials logon option (BETA)
  19. Improved logic to check relative links in SharePoint documents and pages
  20. Added button to start short video with explanation how to use ReplaceMagic