#Moving the TER from mirrors to a CDN Authors: * Steffen Gebert * Markus Klein Comments are welcome! Please add your name here. Keep in mind that we can improve the setup afterwards once we have it running (please don't overcomplicate things). **tl;dr the concept of mirror servers pulling from the main repo should be replaced by a CDN** * The contents of the TER can be viewed here: \url{http://ter.tue.nl/} * There has been an earlier version of this document, which is now discontinued: \url{https://notes.typo3.org/p/deactivating-TER-mirrors} ##New Implementation * As only mirror in mirrors.xml.gz there will be ter.typo3.org, which will be served by the CDN * The new version can be previewed at \url{http://repositories.typo3.org/mirrors-cdn.xml.gz} * The CDN service will be sponsored by Marketing Factory and is based on the Level3 CDN * The edge servers pull ter-backend.typo3.org automatically, if there is no valid local copy * The only thing that we have to care about is purging, when things have to be removed from edges Suggested expires times: * *.t3x -> 1year * *.png, *.gif -> sout1daysout 1year * these are the icons of the extensions? -> yes.. but as they have the version number included, we can also cache it forever..) * /extensions.{md5,xml.gz} -> 1day * Would suggest 4 hours or even less, so you won't need to invalidate it * AFAIK those to files have to be in sync. If a client downlads version X of the .md5 and version X+1 of the .xml.gz, the checksum is invalid. I have no other idea than actively purging it on chnages Purge Actions: * extensions.xml.gz and extensions.md5 * after extension upload to TER * souteverything (alternatively implement resolving the correct path..)sout * soutdeletion from TER in case of security releasesout * extension's .t3x + .gif + .png, when it is deleted * it seems like everybody can delete his/her own extensions ##Action items ### ###typo3.org team * implement purging of CDN files * we therefore need instructions by Marketing Factory (Ingo Schmitt) * Purging is done by * a "invalidation" call to a webservice for a specific URL. This call triggers the invalidation process, which can take some time (All cache nodes have to work on it) * fine. we need that * a "status" call to a webservice gives you an information on the process of the invalidation * extensions.xml.gz and extensions.md5 after extension upload to TER * extension's .t3x + .gif + .png, when an extension is deleted * ensure that our own varnish does not cache any of these files (we could disable caching for /fileadmin/ter/ completely?) ###Server Team * Fix the headers sent by the typo3.org backend server to sent specified expire times * we cannot change these right now without purges, as otherwise the typo3.org varnish would play out stale content * Change mirrors.xml.gz file * Write an announcement article for typo3.org * Contact the mirror sponsors and thank them for their contribution, ask to end the service ###TYPO3 CMS Team * Ensure that the Extension Manager in supported versions of TYPO3 CMS can handle * the change to a new mirror, if the previously selected mirror is removed from the mirrors.xml * I guess we also need to update non-supported versions to be able to handle this !? (Otherwise those installations will not be able to install security updates of extensions anymore) * It depends on, if something really has to be changed. I can imagine that only things happen that e.g. when a specific mirror is selected which is now removed, some issues occur. But basically, installation of extensions (in doubt after manually changing the mirror) should still work fine. * Verify that upload of extensions is not affected (but that should go to \url{http://typo3.org/ter\_wsdl.php} or so..) ###Security Team Comment on the suggestions made above