2017-03-16 Meeting notes
- Dennis Wisnosky
Date
Attendees
Agenda
1) Open Action Items
Proceedings: 
20170316 FIBO FPT
DA progress - on Protege errors - has found the reasons but can't re-put into GitHub. There were 2 issues. One was to do with the import of some definition, that seems to have been a bug in protégé and was able to work around. The 2nd issue was an import relating to loans. Seems that the Protege thing gets lost if the chain of imports gets too long.
Actions list:
OK: Create JIRA ticket. done
JG document how CL will work. The Jenkins job is already working. Omar is using this.
So this is closed ("Works" rather than "documented", seems to be adequate for this)
OK: can't show us progress on the About file part of this yet.
JG: DW still awaiting emails for the new volunteers. Done
Omar: Process: Someone makes a commit in Git. OK reads the comment, links that to a JIRA issue and based on the comment there grabs the Author, and some additional properties and makes a commit into StarDog. Then he updates the JIRA ticket. Maybe also a post build action for archiving this. DW wasn't this action item to do this with real FIBO? OK: Yes but have not got to that yet.
ACTION: OK on mockup needs to be FIBO-Master, not Pizza, by next Tuesday.
DA: to clarify - is this the Jenkins job that feeds things into StarDog? What triggers it?
The Commit. Which Fork and which branch(s) is it working on? OK: right now it is working on OK's Text fork and branch.
DA we need to decide the answers to the governance questions. For instance, it only makes sense to do diffs across approved versions of FIBO, not every Fork and Branch. This only wants to happen in Pink and FIBO-Master in the Pink and main branches in the edmcouncil repository.
Also need to figure out how to make this happen only when we make a Pull Request (or when we approve one?) and not for every commit. So now there are things we as a group need to decide about.
When should this happen? DA proposes only for completed Pull Request against Master and Pink. OK: also the plug-ins may have changes since we last looked. Lots of config to work out on those.
MB: we are still waiting for the other (user) group to meet and decide about what they actually want for content negotiation for non OWL visitors to a link - is this what Omar is talking about here for HTML?
OK: Yes, and is working on something by way of demo but would love to get input from that group about what we want to see. Needs enhanced HTML output.
MB maybe having something by way of a demo will help get those conversations under way.
(this is the calls suggested by PR and David Frankel on the OMG content negotiation requirements. DA: can OK review some of this?
OK: now for HTML there is nothing, but have a text piece showing that the publication successfully goes out there. As a minimum, just having FIBO Master in there so someone has a link to the RDF for now, as a place holder. So, we will have something stood up while we are waiting on the outcome of the content negation requirements discussion (the group for which has not met yet). So we will have a keystone on which to build up whatever the content negotiation WG comes up with.
DA: referring to the Drums diagram: sent a message on Monday about this. Here we have FIBO-Master in the big drum. this contains a copy of FIBO Release (AKA Pink). In GitHub the repositories are called FIBO-Master and Pink. The Big Drum is DirtyPink i.e. the part of FIBO Master that does not include Pink. This does not change, it is reference for FCTs to look at as they build new material that goes into Pink, alongside any other resources they look at.
When they make change to Pink, they will want to make those changes to both at once.
So we need to decide: Will they (a) make changes in Pink and a Jenkins job gets them into Master, or the other way around? The third, absurd option would be to make the same changes to both in parallel.
The things being, we have 2 sets of publications, one being DirtyPink which is not vetted, and we have Pink that has been vetted. So our publish infrastructure works on entire repos. Can have it on a sub set of a Repo as a new requirements for the pub infrastructure.
JG assumes that for whatever branch we have there is one Repo. Then whatever part (branch or tag) will result in a Build being run that publishes in the corresponding name. DA this does not take into account the two entire publications and different pub processes with different expectations for the end user. These are Release, being the vetted stuff, and there is the FIBO-Master in which parts are the unvetted stuff. So for every product there are two publish processes, exactly as shown in the Drums diagram. These are not the same.
This is the whole point. These both get published.
JG: Publishing = accept Pull Request into Master. So if this is OK you pull into the Release branch. Both pulls result in a Jenkins job being launched that publishes it under that name on the http://spec.edmcouncil.org
THE ABOVE IS NOT CORRECT. We have builds on an as they happen basis by accepting a pull request into FIBO-Master. Then, every quarter, we have Publish from FIBO Master of: FIBO-Master and FIBO-Release and the products (FIBO-V, NL Glossary and the latest mapping to Schema.org} into spec.edmcouncil.org/fibo. I have updated the Drum diagram with these words. Our challenge is to figure how to do this, as JG said today.
DA: so then how do you ensure that these are the same. FIBO-Release in the Drums diagram is the branch called "Pink today. JG the branch name should be the same.
So we need to change the name Pink to FIBO-Release. DW: YES! But, this is within FIBO-Master, until it is published.
The question here though is, do we have a single branch and take a sub set when we publish (changing the spec for the Publisher), OR do we have a way of making them in synch. Or rely on people to do the 2 pull requests?
These are 1 decision by one group: something that was in DirtyPink now belongs in Pink, i.e. FIBO-Release. The result must be that the two items (Pink and DirtyPink) are both changed at the same moment - so that the items is deprecated from one and added in the other.
DA is asking: A single decision is made by an FCT; 2 sets of stuff change in lock step when that decision is made. So this is either 2 pull requests (change to current spec), or one PR kicks off the other. We need to identify how we achieve this requirement.
JG are we talking about the exact same change in 2 branches? See Drum diagram: one branch has the core 6 Domains, the other has those plus the others.
Are these 2 different versions of FIBO? Yes, and one is a super-set of the other. JG so if there are 2 different versions of this, there are 2 different pull requests in 2 different branches.
TC do we have to do it that way, or can we have a Jenkins hob that when we have the files pushed into one then we over-write the other branch.
JG but how do you over-write a sub-set of the files and assume that it works? Need test jobs over the resulting graph.
DA one question is which direction does it go e.g. from the one in the bottom, to the top. DW has suggests this is the other way round - so the FTCs' job is to move things from DirtyPink into Pink. Make changes in Pink. These are then forwarded to the FIBO-Release one below.
So these are 2 possible strategies. If we start with having 2 branches in a single Repo.
JG would have to know where the information is stored, which parts are moved from Master to Release. If we start with Release, it's obvious. It is checked into Release as we have all along. We also have to copy files, from one branch to another. that loses the history of them/
TC can we do a kind of forced merge - force it to take all the changes and not try a real merge.
DA has considered we do that - have FCTs do a Pull Req into FIBO-Release (currently Pink), and that happens as now, but also triggers a job that does ether a rebase or a merge so you rebase those changes onto the thing at the top. Then you would have that history. After that rebase your Master branch would look exactly the same as the other branch.
TC you almost don't need to other branch if you have the appropriate filter. Then you only have one branch with everything in it. Then the pub software has to be able to say "here is the sub-set that is published under release and here is the sub=-set that is published as Master". Then there is not synch at all.
TC however there are places where that can go wrong, e.g. if there is a file that was imported and wasn't copied into the release for the others or wasn’t in the list of files for the other - that would then cause a break. At some point, even if we copy to some location where we can run a test over it, to make sure the sub-set is OK. Then OK.
JG also lose the naming thing whereby every artifact can be linked to where you can trace and track it down. So for instance would see a Release thing on the published thing but would not link back.
TC is that an issue - they are just copies of what is above, not being locally edited.
JG so the URL of the class would have the "release" in it but would have to replace with the term "master" or "pink" etc. to find the URL for that.
TC if we do that, is the alternative really easier?
JG yes, because the new thing would not include the Dark Pink stuff as these would show up as deletes.
DA there are 2 scenarios: in one, FCT makes changes to Release. So, then the Merge is appropriate.
JG if we find the state of the ontology in the Artifact Ontology. Then we can use that.
JG as this as a new user story: take account of the state of the domain and generate a different kind of HTML output showing the states, with a zip files people can download for each state.
Would need to still have full lineage for the resource. Then the site would make a distinction between released and not released domains.
EAL for a given Domain there will be multiple states, Release and what the FCT are working on.
Here "Release" means the Git branch called Release. On top of the branch you can set a tag. In the Master branch, every time you need to set a stake in the ground you set a tag, and this triggers another publication in which we would see a version with that tag in its URL.
So Release" is a branch in FIBO - FIBO-Release is what is now called Pink. DW will change that.
DW will not approve of a scenario where an FCT works on Release and then puts that back into Master. Need to look at the whole ecosystem/
JG has an idea. Call Pink "master" and with the standard GitHub semantics of "master". Then master always over-writes the previous version of Master. Then for an official release you set a tag. The publisher can say only those domains that have Release status in the Artifact ontology will be published.
EAL: is everyone then working off one single branch of one master? Yes.
DA the issue if we do this is the Pub software has to be smart enough to publish a sub-set.
JG: so doing a Pull Request doesn't do anything special but when you put a Tag on something then the tag triggers a Jenkins job. the job knows this is a thing, and publishes only those domains that have Release status as defined in the Artifact ontology. Also Sub-domain and also Ontology.
MB internal changes versus moving an ontology from DirtyPink to Pink?
DA: When I have made a change in DirtyPink then it becomes Pink. In the 2 branch story, you would make a change in the top one and promote it to Pink. So changes that are not ready to promote they stay in DirtyPink until we are ready to promote, then at which point the first of the MB scenarios comes into play where the status changes. DA you test locally before promoting. Your testing of the thing you are about to publish is done in the JIRA-named branch. So we test in the JIRA-named branch before doing the tests
DA that's why it is attractive to have a branch with just the thing you are doing - e.g. if you want to add CAE as a new domain, add that in Pink test all 7 domains, make sure they all work together.
Where do these get pushed? Sensibly this is in the stuff at the bottom (FIBO-Release)
The requirement for the FCT is to ensure consistency only with FIBO-Release not all of DirtyPink.
JG you are not supposed to test against the latest Pink but the currently published version of that graph. So you can say I want to run my automated tests on the released material only.
Tests for the released version - are these against the latest Pink, or do you roll back to the Released version of the released material.
MB need to recognize that for something that has Released status, there is new work on that, whereby that work is not at released status but the domain (or sub domain) is.
DA specific example: In the wiki, has published the current hygiene tests for "Well formed OWL" which can apply to DirtyPink such that we can make future improvements to that. It means that we can actually test changes agaist the whole FIBO-Master graph not just the Release(d) domains.
JG GitHub has a new feature called protected branches so we can make the master branch protected. Can have all kinds of different maturity levels or completion status in the other branches, and use the new feature for protected branches, for the master. Also have a sub-set that is the released parts of FIBO that can be tested separately. Then the dev cycle can go faster.
Decisions:
Action items
- @Omar mockup needs to be FIBO-Master, not Pizza, by next Tuesday